I am getting tweets from twitter about a topic and everthing is good according to the code below, but I do not understand why I am getting error KeyError "text"?
client = tweepy.Client(bearer_token, consumer_key,consumer_secret,access_token,access_token_secret)
auth = tweepy.OAuth1UserHandler(consumer_key,consumer_secret,access_token,access_token_secret)
api=tweepy.API(auth)
class StdOutListener(StreamingClient):
def on_data(self, data):
producer.send("twitterstream", data)
a_data=json.loads(data)
print (a_data["text"])
time.sleep(0.5)
return True
def on_error(self, status):
print(status)
I am getting this error
Stream encountered an exception
Traceback (most recent call last):
File "c:/xampp/htdocs/major/kafka_file/stream_data.py", line 26, in on_data
print (a_data["text"])
KeyError: 'text'
This is the data that I am getting
{"data":{"edit_history_tweet_ids":["1635959332016627713"],"id":"1635959332016627713","referenced_tweets":[{"type":"retweeted","id":"1635706425438769153"}],"text":"RT @karishma_behera: Drunk TT pulled her at KJM . While the girl was telling she had her ticket, showed ticket to TT but TT didn't listen…"},"matching_rules":[{"id":"1635553170620563457","tag":""}]}
>Solution :
The error you’re encountering is because the JSON object a_data does not have a "text" key in some instances. To handle this case, you can add a conditional check to verify that the "text" key exists in the JSON object before trying to access it.
Here’s a modified version of your StdOutListener class that includes the necessary check:
class StdOutListener(StreamingClient):
def on_data(self, data):
producer.send("twitterstream", data)
a_data = json.loads(data)
# Check if the "text" key exists in a_data before accessing it
if "text" in a_data:
print(a_data["text"])
else:
print("No 'text' key found in this data")
time.sleep(0.5)
return True
def on_error(self, status):
print(status)
With this modification, your code should no longer raise a KeyError when the "text" key is not present in the JSON object. Instead, it will print a message indicating that the "text" key was not found.
Keep in mind that the structure of the data received from the Twitter API can vary depending on the type of tweet, such as retweets or quote tweets. In some cases, you might need to access the "text" key in a nested JSON object. Make sure you thoroughly review the structure of the data you receive to ensure you’re processing it correctly.