Matthew
Caldari BloodStar Technologies
|
Posted - 2011.07.31 15:22:00 -
[1]
So, if we take it as read that queries against contract/order end date are expensive, but those against start date are reasonable, and that this cannot easily be changed, then the proposed solution seems reasonable. Given this, there seem to be three key design parameters to consider:
Minimum Query Frequency
This is defined by the time window of recently issued orders included in the API response. If the API only includes the last weeks worth of recently issued orders, then the user must query the API at least once every not-quite-a-week to be sure of having seen all records.
Note that the IndividualContract API does not cover this issue, because even if it allows the specified contract to be retrieved, the user has no API method of knowing the missing contractIDs.
So, the question here would be, is it reasonable for a user to end up with holes in their records if they do not query the API at least once every week? What would be a reasonable minimum interval?
Start-Up History Pull
Long-term users of the API can build up a history. But where a new user starts to use the API, or a new API is introduced, the ability to retrieve older historical information is limited.
With the currently proposed API, the ability to build up a history would be limited by the time window of recently issued orders. In this case, you would only be guaranteed a complete history going back one week from the point at which you start to use the API.
This is where Marcel's suggestion would come in. Once a contract/order is completed the record no longer changes. So the API server could archive this information into some sort of static cache, rather than troubling TQ for it every time. We could then get three API calls:
CurrentContracts - as proposed in the dev-blog, non-expired items as well as items issued less than X time ago.
ArchivedContracts - All expired items, ordered by creation date, with a paging mechanism to go back 1+ month. (this would just require filtering by isExpired, rather than ordering by ExpiryDate, which is hopefully a less costly query)
IndividualContract - as proposed in the dev-blog, give a contractID and get that individual contract back.
The new user starting to use the API can then use ArchivedContracts to retrieve an initial "boot-strap" historic dataset. They can then keep it updated using CurrentContracts, filling in gaps with IndividualContract.
The other advantage of ArchivedContracts is that it would act as a failsafe against the Minimum Query Frequency issue above. If the user does miss the Minimum Query Frequency for some reason, they can use ArchivedContracts to recover the missing contract details.
Bulk vs Individual Update of Contract Status
This point is about the efficiency of retrieving updates via CurrentContracts vs having to make a lot of calls to IndividualContract.
For example, if CurrentContracts returns the most recent week of issued contracts, but the typical life of your contract is 10 days, then you are going to have to use IndividualContract to update the majority of your completed contracts, leading to a lot of spamming of the IndividualContract API. In this case it may be more efficient to extend the time interval returned by CurrentContracts.
This is probably something best determined by looking at the distribution of time-to-expiry on all contracts to determine a suitable minimum coverage of CurrentContracts. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |