Follow

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Contact

Getting KeyError: 'text' while trying to classify tweets

I am getting tweets from twitter about a topic and everthing is good according to the code below, but I do not understand why I am getting error KeyError "text"?

client = tweepy.Client(bearer_token, consumer_key,consumer_secret,access_token,access_token_secret)
auth = tweepy.OAuth1UserHandler(consumer_key,consumer_secret,access_token,access_token_secret)
api=tweepy.API(auth)
class StdOutListener(StreamingClient):
        def on_data(self, data):
                producer.send("twitterstream", data)
                a_data=json.loads(data)
                print (a_data["text"])
                time.sleep(0.5)
                return True
        def on_error(self, status):
                print(status)

I am getting this error

Stream encountered an exception
Traceback (most recent call last):
  File "c:/xampp/htdocs/major/kafka_file/stream_data.py", line 26, in on_data
    print (a_data["text"])
KeyError: 'text'

This is the data that I am getting

MEDevel.com: Open-source for Healthcare and Education

Collecting and validating open-source software for healthcare, education, enterprise, development, medical imaging, medical records, and digital pathology.

Visit Medevel

{"data":{"edit_history_tweet_ids":["1635959332016627713"],"id":"1635959332016627713","referenced_tweets":[{"type":"retweeted","id":"1635706425438769153"}],"text":"RT @karishma_behera: Drunk TT pulled her at KJM . While the girl was telling she had her ticket, showed ticket to TT but  TT didn't listen…"},"matching_rules":[{"id":"1635553170620563457","tag":""}]}

>Solution :

The error you’re encountering is because the JSON object a_data does not have a "text" key in some instances. To handle this case, you can add a conditional check to verify that the "text" key exists in the JSON object before trying to access it.

Here’s a modified version of your StdOutListener class that includes the necessary check:

class StdOutListener(StreamingClient):
def on_data(self, data):
    producer.send("twitterstream", data)
    a_data = json.loads(data)

    # Check if the "text" key exists in a_data before accessing it
    if "text" in a_data:
        print(a_data["text"])
    else:
        print("No 'text' key found in this data")

    time.sleep(0.5)
    return True

def on_error(self, status):
    print(status)

With this modification, your code should no longer raise a KeyError when the "text" key is not present in the JSON object. Instead, it will print a message indicating that the "text" key was not found.

Keep in mind that the structure of the data received from the Twitter API can vary depending on the type of tweet, such as retweets or quote tweets. In some cases, you might need to access the "text" key in a nested JSON object. Make sure you thoroughly review the structure of the data you receive to ensure you’re processing it correctly.

Add a comment

Leave a Reply

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use

Discover more from Dev solutions

Subscribe now to keep reading and get access to the full archive.

Continue reading