Pages: 1 2 3 4 5 6 7 [8] 9 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Desmont McCallock
187
|
Posted - 2012.07.11 12:00:00 -
[211] - Quote
Because by using UTC you don't have to worry about DST. |
Muscaat
EVE Markets
12
|
Posted - 2012.07.11 12:09:00 -
[212] - Quote
Desmont McCallock wrote:Because by using UTC you don't have to worry about DST.
Who is "you" in that statement?
If the generators (the people who write the uploaders) are forced to convert to UTC then they do need to worry about DST, because they need to convert away from their local timezone (whatever that is right now).
The spec doesn't currently force UTC, which means that uploaders don't need to worry about DST (other than knowing what timezone they're in right now, which their date library does for them, right?) and consumers just need to parse the date properly and not worry about it (because their date library deals with the timezone for them, right?)
I agree that things would be a lot simpler if everyone just used UTC and got used to going to work at 23:00 or whatever, but I don't think that idea will catch on |
Desmont McCallock
187
|
Posted - 2012.07.11 12:47:00 -
[213] - Quote
Muscaat wrote:Desmont McCallock wrote:Because by using UTC you don't have to worry about DST. I agree that things would be a lot simpler if everyone just used UTC That's the point behind the suggestion.
And just to clarify something here. Both me and Snarf said 'Times should all be in UTC.' as a suggestion. If we said 'All times have to be UTC' (which make it mandatory) then you would have a point to argue.
** Is there a Hermes recursion going on, cause there is a lot of misunderstanding around?
Edit: Btw, as we discussed (and I proved to you) in the Google group discussion, the time value has to be in UTC but you clearly have to state the TZ diff if you are using it. |
Muscaat
EVE Markets
12
|
Posted - 2012.07.11 13:36:00 -
[214] - Quote
Desmont McCallock wrote:Muscaat wrote:[quote=Desmont McCallock]Because by using UTC you don't have to worry about DST. Edit: Btw, as we discussed (and I proved to you) in the Google group discussion, the time value has to be in UTC but you clearly have to state the TZ diff if you are using it.
I'm not quite sure I understand this, but I'm fairly sure we don't actually disagree all that much so I'll stop arguing the point now |
Desmont McCallock
187
|
Posted - 2012.07.11 14:02:00 -
[215] - Quote
In that discussion I am 'Jimi' as I'm known in the EVEMon forums as well. |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.11 14:11:00 -
[216] - Quote
Yeah, this is not the most interesting discussion, so I'll just leave this hear to dispel confusion, myths, or half-truths. Here is basically all time fields in UUDIF are bound by:
Quote:The data/time in ISO 8601 format (example 2011-10-22T15:46:00+00:00)
Follow the ISO 8601 spec and you are compliant with UUDIF. If your local time is UTC, your offset is +00:00. If it's some other timezone, specify it in the tz offset. As long as you're doing that, people with ignorant consumers may get burnt, but that's only because they aren't doing what the spec demands (parsing tz offsets applied to UTC).
As Muscaat or someone else mentioned, your programming language of choice undoubtedly has some modules/packages/libraries for dealing with ISO 8601 dates. Make sure you use them and you'll be all good.
And now for some hopefully more exciting discussions :) EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Desmont McCallock
187
|
Posted - 2012.07.11 14:20:00 -
[217] - Quote
Ilyk Halibut wrote:And now for some hopefully more exciting discussions :) I'll drink an ice tea on that.
|
Schereau Lasomere
University of Caille Gallente Federation
0
|
Posted - 2012.07.11 19:40:00 -
[218] - Quote
Thanks Snarf and Desmont! that clarified things alot.
Desmont McCallock wrote:EMDR is indeed changing [currentTime] (as a forwarder it's allowed to) and its not the same with the 'currentTime' EVEMon generates.
The weird thing is that in the data stream I'm receiving, currentTime keeps jumping back and forth by about a minute:
Quote:currentTime,generatedAt,regionID,typeID,... . . . 2012-07-11T18:40:29+00:00,2012-07-08T20:04:25+00:00,10000002,38,... 2012-07-11T18:41:38+00:00,2012-07-10T20:31:42+00:00,10000042,3436,... . . . 2012-07-11T18:41:38+00:00,2012-07-10T20:31:42+00:00,10000042,3436,... 2012-07-11T18:40:30+00:00,2012-07-11T18:41:04+00:00,10000030,5401,... . . . 2012-07-11T18:40:30+00:00,2012-07-11T18:41:04+00:00,10000030,5401,... 2012-07-11T18:41:39+00:00,2012-07-09T21:12:23+00:00,10000060,1944,... . . . I thought that if EMDR does change the currentTime, then the currentTime would be always increasing, rather than jumping back and forth, when data is downloaded from a single relay. Is the order data passing through two different data processors (PCs) with unsynchronized clocks in EMDR, even though I'm downloading the data from a single relay? The EMD architecture picture in the docs seems to hint at that. Trying to figure out the cause and potential effect of what I'm seeing. |
Desmont McCallock
187
|
Posted - 2012.07.11 19:56:00 -
[219] - Quote
Although it's indeed happening, I wonder why you are so obsessed about it. Are you using the 'currentTime' for something? Could be a side effect of ZMQ queuing but Ilyk is the man with answers to that. |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.11 21:06:00 -
[220] - Quote
Looks like in my hurry to set up the secondary relay, I forgot to install ntp on it. Whoops. Fixed, the drift will correct over itself. EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
|
Schereau Lasomere
University of Caille Gallente Federation
0
|
Posted - 2012.07.11 22:29:00 -
[221] - Quote
Desmont McCallock wrote:Although it's indeed happening, I wonder why you are so obsessed about it. Are you using the 'currentTime' for something? Guess it's mix of curiosity and woriess. I'm not using any of the timestamps yet. I'm still very early in the dev phase. Was trying to figure out which timestamp to use for freshness verification but the EMDR and UUDIF documentation wasn't clear on that in my oppinion.
Guess it's in my nature to find explanations to things that I can't explain to my self.
Since the UUDIF spec doesn't state what source to use for the generatedAt timestamp, I as a potential uploader developer would not know what source to use. Because of that I was worried that some uploaders might use the cache file creation/update time converted to UTC as generatedAt time, which obviously would not be a good idea. That potential problem made me wonder how to do the freshness verification.
|
Muscaat
EVE Markets
12
|
Posted - 2012.07.12 15:05:00 -
[222] - Quote
Desmont McCallock wrote:Ilyk Halibut wrote:And now for some hopefully more exciting discussions :) I'll drink an ice tea on that. Make mine a hot tea, milk and sugar |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.13 19:27:00 -
[223] - Quote
I have added an 'EMDR' upload key with a hash that uniquely identifies each uploader by IP address. See the mailing list post for more details:
https://groups.google.com/forum/#!topic/eve-emdr/WNiN63KJQ_Q
Hopefully this is useful for some of you to blacklist the troublemakers we'll eventually attract (if we haven't already). EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Dragonaire
Corax. The Big Dirty
45
|
Posted - 2012.07.14 05:03:00 -
[224] - Quote
Just to add my 0.02 ISK in here about the generatedAt time. It could also be the time from the API XML if that was the source so keep that in mind before getting to carried away on expecting it to be able to tell you which of two messages was first down to the second or even much below a few minutes. In the other thread that you posted a link to this one and in there was some back and forth about how to do the time and that why we decided on using ISO 8601 and there is also some clarifying explanation at the bottom of the page in the spec on using the complete time format so as to not have any confusion about if it was in local or another time zone. We all felt it was important not to make mistakes that we had all seen when that wasn't done like with the API data/times without any time zone on them etc. You also wondered about the sync of varies sources etc. There can be problems there because in the case of the cache files it could be dependent on the clock of the user uploading the data being set correctly, I believe the Eve client uses the time it gets from the Eve server so it should be accurate but someone that is more familiar with the cache would have to answer that.
So to kind of sum it up the date/time from a single source should be directly comparable but when coming from multiple sources it would take getting to know where they all come up with the date/time reported in generatedAt and insuring in your application you adjust for any differences. Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal PHP API library thread for more information. |
Desmont McCallock
189
|
Posted - 2012.07.14 08:40:00 -
[225] - Quote
I'm really curious what impact the combination of EMDR/EVEMon had on the EVE economy. Looking forward to see the next 'Price Indices' report. |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.18 12:50:00 -
[226] - Quote
ShoGun/Baptiste has volunteered a second french relay to complement Jognu's. This works out well, as there has been quite a bit of usage on the european relays. For those of you in Europe, you may now use the new relay for additional redundancy, or as your sole data provider.
tcp://relay-eu-france-2.eve-emdr.com:8050
Thanks to ShoGun for volunteering it. Here is the full list of relays:
http://www.eve-emdr.com/en/latest/access.html
Greg EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Shellac Brookdale
RAZOR Alliance
6
|
Posted - 2012.07.18 19:25:00 -
[227] - Quote
Ilyk, Desmont -- Great work so far!
But can we please have something like this for killmails too? Have EVEmon users auto upload killmails into a separate data relay network and enjoy all kinds of new killmail consuming apps making use of this? Making it easier for killboards to sync kms? That would be huge.
|
Desmont McCallock
193
|
Posted - 2012.07.18 21:03:00 -
[228] - Quote
Looking into the possibility of such a project, I began scanning the cache files (with EVECacheParser) to see if the kill mails info can be retrieved. I discovered that the kill mails are indeed cached under the method 'charMgr.GetRecentShipKillsAndLosses'.
The data that can be retrieved are: killTime, iskLost, killID, solarSystemID, victimCharacterID, victimCorporationID, victimAllianceID, victimFactionID, victimShipTypeID, victimDamageTaken, finalSecurityStatus, finalCharacterID, finalCorporationID, finalAllianceID, finalFactionID, finalShipTypeID, finalDamageDone, moonID, warID, killBlob (which contains info about the attackers and the items dropped).
Now, if we can agree to a unified format much like we did with the market data, I don't see any problems in creating an EKDR (EVE KillReport Data Relay).
Edit: Prerequisite for this to work is that a player has to open the combat log in the character sheet in order for the cache file to be created, much like it works with the market data. |
Shellac Brookdale
RAZOR Alliance
6
|
Posted - 2012.07.18 21:57:00 -
[229] - Quote
Wouldn't retrieving the killmails trough the API be an option for clients to push data? |
Desmont McCallock
193
|
Posted - 2012.07.19 06:56:00 -
[230] - Quote
I thought that you might ask. Killmail API is defected as hell. If you have given to a service (site or client) an API key to retrieve kill mails, you can't retrieve the same info (even with another API key) so to use to another service. Meaning that the killmail API call doesn't return persisting data.
This is also the reason I haven't added a kill mail monitor in EVEMon yet. Those that use any API key to feed a killboard won't be able to use EVEMon (or any other app/service that use the same API call) to monitor the kill mails and vice verse. |
|
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.23 15:47:00 -
[231] - Quote
Shellac Brookdale wrote:... But can we please have something like this for killmails too? Have EVEmon users auto upload killmails into a separate data relay network and enjoy all kinds of new killmail consuming apps making use of this? Making it easier for killboards to sync kms? That would be huge. While this would certainly be useful, I am just stretched way too far to undertake it myself. This would need to be a separate entity from EMDR, but a huge chunk of the codebase could be re-used trivially. Basically the only thing that would need changing would be the gateway daemon. Instead of looking for market data, it'd need to use whatever format the uploads come over as.
I'd love to offer advice and pointers for anyone wishing to undertake this, so poke me if you do end up giving it a shot. The EMDR codebase and ZeroMQ are both relatively simple (and BSD-licensed, extremely permissive), and much of the tedious testing and messing with relays/announcers has been done for you (those could both be re-used without modification). An adaptation of EMDR to work as a new service could be done over a weekend of solid effort.
EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.07.23 15:48:00 -
[232] - Quote
EMDR has added relays in Denmark and Holland, courtesy of Karbowiak:
http://www.eve-emdr.com/en/latest/access.html EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Qualla Amborante
Mindelan Heavy Industry
0
|
Posted - 2012.08.03 08:49:00 -
[233] - Quote
The Holland relay does not work, does not accept the connection. Gemany works fine.
Thnx. |
Qualla Amborante
Mindelan Heavy Industry
0
|
Posted - 2012.08.03 10:44:00 -
[234] - Quote
Hello,
Can I tell the feed somehow that I do not want the historical data? If I get all data, and save it, I do also have this history, and save a load of bandwith, or am I mssing something?
Qualla |
Kaladr
Dreddit Test Alliance Please Ignore
36
|
Posted - 2012.08.04 02:47:00 -
[235] - Quote
Desmont McCallock wrote:I thought that you might ask. Killmail API is defected as hell. If you have given to a service (site or client) an API key to retrieve kill mails, you can't retrieve the same info (even with another API key) so to use to another service. Meaning that the killmail API call doesn't return persisting data.
This is also the reason I haven't added a kill mail monitor in EVEMon yet. Those that use any API key to feed a killboard won't be able to use EVEMon (or any other app/service that use the same API call) to monitor the kill mails and vice verse.
I'd love to process the killmail data in the future. However, in many ways, killmails are more of a state syncing problem instead of a pure stream. You can build a whole view of the market by watching the EMDR feed for awhile, but you can't build a whole "view" of killmails. In many ways RSS is a good concept, but it doesn't really scale.
(And to get on the topic of EMDR, EVE-Central is now relaying EMDR data, unofficially at emdr.eve-central.com:8050, and officially once the docs get updated). Creator of EVE-Central.com, the longest running EVE Market Aggregator |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.08.04 19:28:00 -
[236] - Quote
Sorry for the delay, Kaladr's west coast relay is now officially listed here:
http://www.eve-emdr.com/en/latest/access.html EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.08.24 20:01:00 -
[237] - Quote
We're in need of an announcer machine. If you'd like to volunteer one and have a good chunk of bandwidth to spare, please see: https://groups.google.com/forum/?fromgroups=#!topic/eve-emdr/JXMr5iqZGbU EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Vaerah Vahrokha
Vahrokh Consulting
1949
|
Posted - 2012.10.31 08:14:00 -
[238] - Quote
Hello,
I'd like to attach to EMDR and fetch the data but I have a problem: my hosting won't let me install daemons or perma-running processes that can sit listening to sockets and similar. I can only run a cron.
Is there any way I can deal with this restriction and still get data?
I noticed that the peeps at Element43.com seem to be hosted on GoDaddy yet they can process EMDR feeds, so there seems to be a way.
Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
9
|
Posted - 2012.11.01 16:49:00 -
[239] - Quote
Vaerah Vahrokha wrote:Hello,
I'd like to attach to EMDR and fetch the data but I have a problem: my hosting won't let me install daemons or perma-running processes that can sit listening to sockets and similar. I can only run a cron.
Is there any way I can deal with this restriction and still get data?
I noticed that the peeps at Element43.com seem to be hosted on GoDaddy yet they can process EMDR feeds, so there seems to be a way.
There's not a great way to do this. Your only hope would be to do some pretty hacky stuff with a long (permanently) running cron entry, but GoDaddy would probably get irate with you.
Element43 isn't on GoDaddy. EVE Market Data Relay - A real-time feed of EVE Market data http://www.eve-emdr.com |
Vaerah Vahrokha
Vahrokh Consulting
1953
|
Posted - 2012.11.02 10:55:00 -
[240] - Quote
Ilyk Halibut wrote:Vaerah Vahrokha wrote:Hello,
I'd like to attach to EMDR and fetch the data but I have a problem: my hosting won't let me install daemons or perma-running processes that can sit listening to sockets and similar. I can only run a cron.
Is there any way I can deal with this restriction and still get data?
I noticed that the peeps at Element43.com seem to be hosted on GoDaddy yet they can process EMDR feeds, so there seems to be a way.
There's not a great way to do this. Your only hope would be to do some pretty hacky stuff with a long (permanently) running cron entry, but GoDaddy would probably get irate with you. Element43 isn't on GoDaddy.
Thank you.
I am already invested in a certain web hosting company (no daemon support sadly) for other websites so all the EvE software and websites - I make all for free - are going to be a pure additional expense to me.
Therefore I'd love if you or some other experienced programmer in there could suggest me a cheap hosting service that lets me install such a permanently running application needed to listen to ZeroMQ messages. I have checked Webfaction and Dreamhost but both are expensive for totally free EvE projects. Plus they really push into multi-year subscription purchases and I can't commit to such a budget. All I needed to do is to listen for EMDR messages 24/7 and slap them in a database, I don't even need a web frontend (no I can't do it with a computer at home). Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
|
|
|
|
Pages: 1 2 3 4 5 6 7 [8] 9 :: one page |
First page | Previous page | Next page | Last page |