Splitting https://dev.twitter.com/discussions/3360 into two threads:
From @knexsy:
"Searches using geocode parameter not working correctly. Appears to have suddenly stopped behaving correctly in the last 48 hrs or so. A result from the API such as http://search.twitter.com/search.json?q=&geocode=53.2739084,-7.4945478,50mi&rpp=100 gives only a fraction of the results the same query would normally give (ie. the 100 results returned are spread over a longer time period that should be the case if all relevant results were returned for the most recent 100). A comparison with the web search http://twitter.com/#!/search/realtime/near%3A%22tullamore%2Cireland%22%20within%3A50mi shows the latter provides significantly more results. (lat, lng are for the indicated place). Note: the search API with the above parameters has been working fine over several months for us - this is a sudden change in the expected results for the same query."
"Specifically - if I obtain via the API http://search.twitter.com/search.json?q=&geocode=53.2739084,-7.4945478,50mi&rpp=100 and parse it out I get 100 results that stretch back in time as far as 12 hours - ie, your result indicates only 100 tweets in a twelve hour period for a radius of 50miles. The following query (via browser for eg) twitter.com/#!/search/realtime/near%3A"tullamore%2Cireland" within%3A50mi (which is the same approx location ) - gives 100 results spreading back over approx 45minutes - ie. a much larger, and more realistic volume. Prior to the 1st Nov this is the result we would have been getting with the search API query (the first query above). We have been using the same (& similar queries) successfully for months to now find they are all broken."
"By the way, the following web search http://twitter.com/#!/search/realtime/geocode%3A53.2739084%2C-7.4945478%2C50mi mimics the results of the first query above and is therefore also incorrect in its results (less that 10% of expected results)."
"In summary - the volume of tweets showing up with the query http://search.twitter.com/search.json?q=&geocode=53.2739084,-7.4945478,50mi&rpp=100 is but a tiny fraction of the results that would have showed up in the query prior to the 1st Nov. - and we have been running queries such as this for over 6months so we know the expected volumes of tweets that are appropriate for various locations throughout Ireland."
"The issue is also clearly visible in @DanielAtik site using http://metaki.com"
"I appreciate it may be difficult to deal with all these various geocode problems, but if you can tell me why the following 2 browser requests should provide 2 very different results in terms of tweet volume considering they relate to the same place & radius, it might help.
https://twitter.com/#!/search/realtime/geocode:53.2739084,-7.4945478,50mi
vs
https://twitter.com/#!/search/realtime/near:%22tullamore,ireland%22%20within:50mi"
Replies
Thanks for your efforts in trying to resolve this.
Is there any difference in volume if you use km instead of miles?
No, doesn't appear to make a difference.
Just pop these 2 versions in a browser and notice time stamps - the first gives much lower volume (as per broken API).
https://twitter.com/#!/search/realtime/geocode:53.2739084,-7.4945478,150km
vs
https://twitter.com/#!/search/realtime/near:%22tullamore,ireland%22%20within:150km
In my testing, I have not noticed a significant difference in tweet volume using km vs miles. I say significant because I'm allowing for the time difference between queries for km and mi based radius'.
What does seem to be happening is that searches are returning tweets from any geo areas (as defined by the 4 geopoints that can be attained using the GEO api) that intersect with the radius that is defined in our search queries.
Let us know if we can be of assistance!
Has anyone been tracking using place_id over a span of time? Any information about place_id volume would be good to have as well.
Hi all!!
How you doing? I'm experiencing the same problem since many weeks ago. :(
As @kurrik said, in http://metaki.com you may check the query related to geotweets and the erratic results.
Any help will be appreciate!! :D
Thanks!
Hi,
I'm also experiencing the same problem returning the location of geotweets. Up to several weeks ago we would get many results returned by our searches but recently we get no results returned. This has been very frustating for us.
Has any progress been made on resolving this issue?
Thanks,
Mike
No, we have engineers looking into these issues but there is no news at this time. Please continue reporting cases where the API returns unexpected volumes of Tweets.
Also, place_id volume data would be helpful.
This issue started on the 1st of Nov. Since that date the volume is < 10% of the expected volume. Can you not check your changelog and determine the upgrades/changes made on this date that would cause the problem. One other thing that changed at the same time is the timestamp, which used to be local time -1hr, now it is local time.
Hi,
I am fetching tweets using the geocode parameter and parsing the ones that come back with geo information, to place them on a map. I am also getting strange values. Here is what is happening to me:
I have, lets say 20 geo tweets in a 1km radius around me, but when I make the same geocode search for 4km, I get.... one geo tweet only.
Also, from those 20 geo tweets around me, 2 of them talk about the 'iphone', so I perform the same geocode query but with "&q=iphone". I get only one different tweet back and it has no geo information.
facing similar issue.
Example on search around Singapore:
radius 100km, does not return anything
http://search.twitter.com/search.json?q=%22special+olympics%22&rpp=100&page=1&result_type=recent&since=2012-01-03&until=2012-01-06&geocode=1.3520830,103.8198360,100km
radius 900km returns same as search on web near:singapore, but singapore' actual radius is max 100km.
http://search.twitter.com/search.json?q=%22special+olympics%22&rpp=100&page=1&result_type=recent&since=2012-01-03&until=2012-01-06&geocode=1.3520830,103.8198360,900km
Have you somehow changed how you parse radius parameter?
One anomaly related to reverse_geocodes, that I found while debugging Singapore specific searches - SG geocode does not translate to SG:
https://api.twitter.com/1/geo/reverse_geocode.json?lat=1.352083&long=103.81983600000001&accuracy=100000 <- this does not return Singapore as on of the results. But it does when accuracy <= 20000 (20 km)
It still does not match singapore, when accuracy >= 900km. Although as last post showed, 900 seemed to be somewhat magical for one query.
Reverse geocode seems to work ok for eg. Estonia, NYC
https://api.twitter.com/1/geo/reverse_geocode.json?lat=58.5952720&long=25.0136070&accuracy=1000000
https://api.twitter.com/1/geo/reverse_geocode.json?lat=40.714353&long=-74.00597299999998&accuracy=1000000
Please start a new thread for this issue.
A fix was pushed which should address this issue, at least it appears to with respect to the following two searches:
https://twitter.com/#!/search/realtime/near%3A%22tullamore%2Cireland%22%20within%3A50mi
https://twitter.com/#!/search/realtime/geocode%3A53.2739084%2C-7.4945478%2C50mi
Both searches seem to return similar quantities of data when "all" tweets are selected.
Feedback and observations are welcome.
I just checked both these queries in a browser, and they do not compare in results.
https://twitter.com/#!/search/realtime/near%3A%22tullamore%2Cireland%22%20within%3A50mi
gives a time span of about 3 min for the first 20 results,
https://twitter.com/#!/search/realtime/geocode%3A53.2739084%2C-7.4945478%2C50mi
gives a time span of about 5 hours for the first 20 results..ie. the number of results returned is <<< than the former.
http://search.twitter.com/search.json?result_type=recent&geocode=51.502,0.0192,4km&rpp=100
Such a query would previously (a couple of months ago) have consistently provided a few hundred results (over several result pages) that fell strictly within the radius provided. At present it still seems to be returning a fraction of the results which in turn are falling within a more generalized area, perhaps approximated to an admin region like the parent city for example.
Hi, I'm also still encountering the problem.
When using the API
https://search.twitter.com/search.json?rpp=100&geocode=32.06616,34.77782,5km&page=1&since=2012-01-16
Out of 100 results, only 1 has a geo location
When using the web search
https://twitter.com/#!/search/realtime/near:%22tel%20aviv,israel%22%20within:5km
At least 50% of the first 100 results have geo location.
There's a large delta.
Is there any progress on this issue?
Thanks
Months roll bye, and nothing changes..makes you wonder about integrity of other data streams
Arne. Any news yet for the issues raised with geo location. I am having a similar problem - none of the tweets matched the location I mentioned. It will be nice if some one working on this can put a short status message regarding the progress to keep us informed.
This isn't a discussion about geolocated tweets matching the specified location, this is tracking the issue that the volume of Tweets for a given location are lower than expected. See https://dev.twitter.com/discussions/4004 if you're having issues with results falling out of a radius.
It's not just that they are lower. Its that 2 similar queries, one via API one via website give significantly different results for the 'same' query.
Again:
"Specifically - if I obtain via the API http://search.twitter.com/search.json?q=&geocode=53.2739084,-7.4945478,50mi&rpp=100 and parse it out I get 100 results that stretch back in time as far as 12 hours - ie, your result indicates only 100 tweets in a twelve hour period for a radius of 50miles. The following query (via browser for eg) twitter.com/#!/search/realtime/near%3A"tullamore%2Cireland" within%3A50mi (which is the same approx location ) - gives 100 results spreading back over approx 45minutes - ie. a much larger, and more realistic volume.
It means that results via the API for geocode are nonsensical and makes one question the reliability of the API in its entirety.
We've made some improvements to resolve this particular problem. If you've been experiencing this issue, could you let us know how the results are looking now?
Thanks!
These improvements seem to have resolved the problem for me, I'm now getting reasonable numbers of tweets from the actual position & radius I requested, will keep testing though.
Lets hope it works as well for other developers.
Thanks!
Looks much better now!!! All geocode search results now look as expected.Thanks!
Hello everyone,
I am experiencing this particular problem with my application.
Please compare these two searches:
http://search.twitter.com/search.json?q=AtascoEn&rpp=100&result_type=recent
---> 3 results, two of which have coord= 40.4466, -36656
Now same search, restricted to these coordinates
http://search.twitter.com/search.json?q=AtascoEn&rpp=100&result_type=recent&geocode=40.4466,-3.6656,5km
---> 1 result
Can someone advice?
Thanks
I'm looking into this issue now -- there looks to be something odd going on.. here's your query refined a little bit to demonstrate in isolation for as long as it's in the index.
http://search.twitter.com/search.json?q=AtascoEn&rpp=100&result_type=recent&since_id=236414906876977150&max_id=236414906876977155
http://search.twitter.com/search.json?q=AtascoEn&rpp=100&result_type=recent&since_id=236414906876977150&max_id=236414906876977155&geocode=40.4481,-3.6596,5km
@episod we are having a similar problem mapping tweets in the Tampa Bay area- our search http://search.twitter.com/search.json?q=rnc&result_type=recent&include_entities=true&lang=en&geocode=27.945386,-82.457085,50mi&rpp=100&callback=? returns geo:null for every tweet, even ones we know are geocoded b/c we tweeted them ourselves from geo-enabled devices. Will probably have a much higher volume in this area during the RNC week after next (tested our mapper around the London Olympics sites), but we're worried those will still come across as geo:null. Any advice? Thank you.
Thanks for more examples, I'll pass these along.
If you're interested in capturing tweets from an upcoming event (rather than one that already happened) you would be MUCH MUCH MUCH better served by using a streaming API instead.
Can you share a specific tweet ID that contains geo tags but has the value as null on the search API?
Here is a tweet ID: 236111571884072960. However, when I do a search looking w/in a geo area the tweet doesn't even show up. Same search term (my username), no geocoded tweets. http://search.twitter.com/search.json?q=darlacameron&include_entities=true&lang=en&geocode=27.7730556,-82.64,50mi&rpp=100&callback=?
(Sorry for the response delay and thanks for your help!)
Thank you, I will be checking this post regularly.
I'm also seeing this issue. geo is null for every tweet here in Vancouver, and it's been like that for a few days now. If you go to twaddler.ca, you can see that it's also happening in larger cities like London and Toronto. I'll also be checking this thread regularly, hopefully this gets fixed up soon!
Worked for me a few weeks ago but I would not gamble my life on it working next time I retest.
Still getting tweets from about 5 days ago with OK geo but since then it's all null. Who changed what about 5 days ago to cause the re-occurance of this issue?
Just a remark from my side: The issue I observed is not about geo being null.
Tweets with correct geo values are not being retrieved by a search which precisely contains those geo values as a search parameter. Removing the geocode parameter from the search clause, brings the tweets correctly (and right values in the geo property).
Seeing the same thing as Alfonso... maybe this is related to the cards rollout? Anyone getting any info from Twitter?
I'm facing just same problem also.. About 5 days ago, I could catch any latitude/longitude by "geo" key from twitter itself and 3rd party apps like foursquare. But this week I noticed there were geo:null by same request... I need any info, too!
Yes I have similar issues. If I search with a geocode parameter then it doesn't return the tweets with geocode data, without the geocode parameter it returns those tweets. This was not an issue a couple weeks ago!
You can now follow this issue while we work to resolve this:
Perhaps my issue is different but my geocode input does return relevant results:
finished Wobulator course at UCSC-Extension
Today I seem to be getting the tweets correctly by using the geocode parameter.
Thanks very much.
Hello,
I am not able to get tweets for most of the geocodes.
I am using
https://api.twitter.com/1.1/search/tweets.json?q=&geocode=38.8781157,-122.398720,1mi&count=100
and it returns the following ( no tweets). This happens for most of the geocodes
{"statuses":[],"search_metadata":{"completed_in":0.033,"max_id":3.7640457958985e+17,"max_id_str":"376404579589849088","query":"","refresh_url":"?since_id=376404579589849088&q=&geocode=38.8781157%2C-122.398720%2C1mi&include_entities=1","count":100,"since_id":0,"since_id_str":"0"}}
Can you please help me.. should i have to specify some other parameter ?
Thanks a lot!
Try increasing your radius beyond 1 mile... there might not be any Tweets geotagged at quite so fine a resolution there.