Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 1 post(s) |
Bob Niac
freelancers inc Territorial Claim Unit
0
|
Posted - 2011.09.23 23:40:00 -
[1] - Quote
Why is the API polling all of its data directly from TQ? Isn't there a way to do a mini-dump of sorts to a database that is optimized for html/php/whatever voodoo you guys use? Not all of the info that is called needs to be real time.
I mean:
I tell program E to get data N. E queries a proxy server (P) for N P asks E for credentials, and succeeds in auth. P then asks TQ for N. N is converted to a usable format, and is cached on P for later use. P hands off N to E. ??? Profit?
I would imagine a CPU limited poll of the TQ database would be going on in the background anyway, to update P. P would also get a full incremental dump during downtime. Err .. I think that works? |
Johnathan Roark
The Graduates Morsus Mihi
5
|
Posted - 2011.09.24 04:23:00 -
[2] - Quote
What makes you think the api is querying a live database? Have you ever noticed that the api servers are available during downtime and often during patch deployments? I think its been mentioned by devs that its a replication of the database. EVEVERIFY - A recruiting API Verification and Audit Tool
Also try out Yapeal for your php api needs |
Bob Niac
freelancers inc Territorial Claim Unit
3
|
Posted - 2011.09.25 15:43:00 -
[3] - Quote
Original dev blog thread from last month stated the api wasn't an optimized database. |
Fishsticks Fred
Dreddit Test Alliance Please Ignore
11
|
Posted - 2011.09.25 21:23:00 -
[4] - Quote
Johnathan Roark wrote:What makes you think the api is querying a live database? Have you ever noticed that the api servers are available during downtime and often during patch deployments? I think its been mentioned by devs that its a replication of the database. Pretty sure it's not available during patches, at least not the big expansion ones. |
Dragonaire
Corax. PURgE Alliance
10
|
Posted - 2011.09.25 23:53:00 -
[5] - Quote
It is sometimes during most of the patch time but we also usually don't get it back for a day or two afterwards as well until they are sure everything is ok in game before turning the API back on. It all depends on the patch. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
|
CCP Prism X
C C P C C P Alliance
134
|
Posted - 2011.09.26 08:56:00 -
[6] - Quote
The API does query TQ-DB proper. It then uses heavy caching strategies to avoid fetching the same data.
There were multiple discussions to move it over to our archives for, at least, archived data and only query TQ for recent updates to the simulation but we never got any further than just talking about it when I was on the team with Stillman. I doubt there are any talks about it right now.
The API used to really **** me off when it came to DB scraping. I imagine that is why they decided to put a DB programmer into C# web service programming. First thing I did was implement the caching I mentioned earlier and now the API is a pretty cool dude by me. Given any run of TQ the API is hogging around 10%+-2% of the used CPU resources (that does not mean it actually uses 10% of our entire CPU resources). This number also applies to reports I've taken when certain, much used, caches are broken so they always hit the DB. In those cases we see abnormal execution counts, pages read from the service.. but the CPU utilization doesn't really suffer for it. The calls are all quite basic relational queries and ordered on the cluster so CPU isn't much of an issue. Of course in the cases of broken caches the API can start shared-locking up resources way more aggressively than it should preventing writes, but that's a worst case scenario. Up until now only inconsequential caches that store completely static data whose tables never actually need an exclusive lock, except on our content authoring servers, have broken like that.
That's this weeks Monday DB ****, kids! ~ CCP Prism X EVE Database Developer "Prism X is my first world problem." ~ CCP FLX If anything in this post was informative or could be considered as 'good news' to you - chances are you've misread it. |
|
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |