Pages: [1] 2 3 4 5 6 7 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.03 01:59:00 -
[1]
Hi,
I've been sitting on this for quite while, but only recently CCP gave the green light for releasing what has been my pet project for many years.
So, I hereby proudly introduce....
RΞVΞRENCE REVERse ENgineered Cache Explorer
This is a strictly x86/x64 Python 2.6 library (Python 3.x support maybe in the future), targeted mainly at developers using Windows, although it has been tested successfully on Linux.
FEATURES
- High-performance iterative cache/bulkdata decoder.
- 100% compatibility with all bulkdata, cache and settings files.
- Programmatic access to data tables.
- Transparent loading of bulkdata on accessing tables.
- Simultaneous handling of data from multiple EVE installations/versions.
- Container classes for all data items found in cache and bulkdata.
- Offline RemoteSvc calls, provided the relevant cachefiles exist. Note that this software DOES NOT interact with the EVE Online client or server.
- EmbedFS (.stuff) file support.
- Various EVE related utility functions and constants.
The library provides programmers with an interface very similar to what EVE uses itself under the hood (yes, CCP reviewed the code and actually didn't mind it), with some variations where I deemed necessary for usability, efficiency, etc, as EVE was never designed to operate on the dataset of multiple EVE installations at once.
It's licensed under the terms of the BSD license so you can do almost anything you want with it (if you're mad enough to port it to other languages, let me know!)
DOWNLOAD The project is available on github here: http://github.com/ntt/reverence
There's a windows installer and source distribution in the downloads section.
USAGE Some examples have been provided (see link above), and some instructions are in the README.txt in the distribution. You'll have to excuse me for the lack of depth of the documentation, but this has been a fairly rapidly changing project over its lifetime (it 'settled' only recently). I'll probably write more thorough documentation SoonÖ.
I hope you'll enjoy this thing as much as I did creating it.
THANKS Special thanks go to CCP for granting permission to release this product, even though it is borrowing heavily from EVE Online's design.
_
Got Item? | EVE API? |
Arkady Sadik
Minmatar Gradient Electus Matari
|
Posted - 2009.10.03 02:37:00 -
[2]
Thank you very much, Entity. This is awesome.
Also, thank you very much, CCP, for allowing this to be published. Not many companies would do that.
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.03 04:06:00 -
[3]
Note that there currently appears to be an issue when attempting to load the entire CachedMethodCalls folder (may cause python to crash), so there's something in there that doesn't play nice. I'll get on that later. For now just don't try to load that particular folder.
This isn't something normally done anyway, so it will not affect most people at all :) _
Got Item? | EVE API? | Cache? |
Kaladr
Amarr
|
Posted - 2009.10.03 05:56:00 -
[4]
Thanks a lot for this Entity. May Reverence and libevecache pave the way toward a more glorious future
(I'll have to borrow some features of course ;)) ---- EVE-Central.com - Your EVE economic and corporate site, featuring the inter-region market browser |
Chyzwar Amarr
|
Posted - 2009.10.03 10:51:00 -
[5]
Hmmm very interesting stuff here but:
running install running build running build_py running build_ext building 'reverence._blue' extension error: Unable to find vcvarsall.bat
I suppose it was because of the absence of visual studio compiler, right?
|
Robacz
Essence Enterprises
|
Posted - 2009.10.03 11:38:00 -
[6]
Thank you for releasing this, best MD post ever.
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.03 13:36:00 -
[7]
Edited by: Entity on 03/10/2009 13:41:14
Originally by: Chyzwar Amarr Hmmm very interesting stuff here but:
error: Unable to find vcvarsall.bat
I suppose it was because of the absence of visual studio compiler, right?
Yeah, use the installer, or download MS VC++ 9.0 Express (it's free anyway)
Originally by: Kaladr (I'll have to borrow some features of course ;))
On a side note: Reverence is a play on Reference (intentional) _
Got Item? | EVE API? | Cache? |
Rilwar
BlackStar Industrial
|
Posted - 2009.10.03 13:56:00 -
[8]
This = The AwesomeSauce v2.0
It's part amazing, and part holy ****. \o/
If you didn't already own every ISK in EVE I'd send you some =x
-------------------------------------------------
Mitnal was here. |
Vessper
SI Radio
|
Posted - 2009.10.05 20:59:00 -
[9]
Nice work!
Just been playing around with it and found an issue but I couldn't find how to post an issue on the website without subscribing to some plan!
Anyway, the calculation of the cache location in Windows is incorrect when using folders containing a space (e.g. Program Files). This seems to be resolved by altering the cache.py file to state:
if os.name == "nt": cacheFolderName = root.lower().replace(":", "").replace("\\", "_").replace(" ","_")
- - - - - - EveHQ Character App |
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.05 22:47:00 -
[10]
Originally by: Vessper Anyway, the calculation of the cache location in Windows is incorrect when using folders containing a space (e.g. Program Files). This seems to be resolved by altering the cache.py file to state:
if os.name == "nt": cacheFolderName = root.lower().replace(":", "").replace("\\", "_").replace(" ","_")
Fixed, thanks _
Got Item? | EVE API? | Cache? |
|
Matthias Dete
Oursulaert Industries
|
Posted - 2009.10.08 19:51:00 -
[11]
So... I've installed Python and Reverence...
Can someone explain to me where it's installed and how to get it to work?
Thanks
---------------------------------------------- Sig: Umbra Congregatio is now recruiting!!Check here
|
Matthias Dete
Oursulaert Industries
|
Posted - 2009.10.08 20:17:00 -
[12]
Hmm... kinda got it working (was problems with the cache not being in the right place in Windows 7)
When iterating over the invTypes however, now and then i get:
UnicodeEncodeError: `charmap` codec can't encode character u'\u2013' in position 8: character maps to <undefined>
Any suggestions... ---------------------------------------------- Sig: Umbra Congregatio is now recruiting!!Check here
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.09 01:46:00 -
[13]
Originally by: Matthias Dete Hmm... kinda got it working (was problems with the cache not being in the right place in Windows 7)
When iterating over the invTypes however, now and then i get:
UnicodeEncodeError: `charmap` codec can't encode character u'\u2013' in position 8: character maps to <undefined>
Any suggestions...
Uh, names and descriptions can contain unicode characters. if your means of displaying cant handle it (such as cmd.exe console), you get that sort of error.
_
Got Item? | EVE API? | Cache? |
LarcatOfRens
|
Posted - 2009.10.19 02:43:00 -
[14]
Is this useable for a non IT-Person to extract market history date?
High/Low/Volume/Average.
-Larcat
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.26 02:07:00 -
[15]
version 1.0.1 is now available (see OP). This version fixes the aforementioned crash bug.
_
Got Item? | EVE API? | Cache? |
Tonto Auri
Vhero' Multipurpose Corp
|
Posted - 2009.10.26 22:47:00 -
[16]
I'd like to ask simple question... could it allow me to write settings files back so EVE could read them? I *really* hate the times when CCP throwing away my time spent on setting up interface, I want some way to keep it intact through CCP's carelessness. -- Thanks CCP for cu |
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.10.28 04:10:00 -
[17]
Originally by: Tonto Auri I'd like to ask simple question... could it allow me to write settings files back so EVE could read them?
Nope. This library doesn't and won't include write capability, as that's a bit dodgy (EULA forbids modifying game files). I would recommend simply backing up your settings files instead :)
_
Got Item? | EVE API? | Cache? |
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2009.12.19 15:11:00 -
[18]
Oddly enough nobody seems to have issues with this toolkit?!
How about feature suggestions then? :P
_
Got Item? | EVE API? | Cache? |
Catari Taga
Centre Of Attention Rough Necks
|
Posted - 2009.12.19 16:32:00 -
[19]
Edited by: Catari Taga on 19/12/2009 16:33:45
Originally by: Entity Oddly enough nobody seems to have issues with this toolkit?!
How about feature suggestions then? :P
It probably does not have many users. I tried it and it worked fine and indeed without issues, great job!
If you are looking to do more work on it how about a function to dump entire tables in datadump compatible sql/mysql format? That's probably what most people want from it anyway.
|
Cyaxares II
|
Posted - 2010.02.11 23:35:00 -
[20]
Extracting market history data with reverence Linkage.
|
|
Femaref
Armageddon Day
|
Posted - 2010.02.14 01:11:00 -
[21]
Originally by: Catari Taga Edited by: Catari Taga on 19/12/2009 16:33:45
Originally by: Entity Oddly enough nobody seems to have issues with this toolkit?!
How about feature suggestions then? :P
It probably does not have many users. I tried it and it worked fine and indeed without issues, great job!
If you are looking to do more work on it how about a function to dump entire tables in datadump compatible sql/mysql format? That's probably what most people want from it anyway.
If you'd check around the examples, there already is one doing exactly that.
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2010.04.02 12:42:00 -
[22]
Anti-thread-expire bump :)
I may have to release an update soonÖ with the planetary interaction stuff coming in...
_
Got Item? | EVE API? | Cache? |
Catari Taga
Centre Of Attention Rough Necks
|
Posted - 2010.04.02 13:44:00 -
[23]
Edited by: Catari Taga on 02/04/2010 13:45:06 Hadn't seen this post:
Originally by: Femaref If you'd check around the examples, there already is one doing exactly that.
Not quite, I was proposing a datadump compatible format, which the example does not quite provide (you would basically need to add a static table to the tool with the current datadump format and the relationships and keep that updated after every change to the structure of either the dump or the game data). Also there is a bug with the example in that it does not add the ";" terminator after each insert.
Originally by: Entity Anti-thread-expire bump :) I may have to release an update soonÖ with the planetary interaction stuff coming in...
Since you bumped it, I always meant to post an issue I had with making it work on non-TQ installs because at least on my system the cache folder is different than what your program was expecting. To make it work I replaced line 94 in cache.py with the following:
if servername == "tranquility": cachepath = os.path.join(path_buf.value, "CCP", "EVE", cacheFolderName, "cache") else: cachepath = os.path.join(path_buf.value, "CCP", "EVE", cacheFolderName + "_" + serverip, "cache")
|
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2010.04.02 13:56:00 -
[24]
Originally by: Catari Taga
Originally by: Entity Anti-thread-expire bump :) I may have to release an update soonÖ with the planetary interaction stuff coming in...
Since you bumped it, I always meant to post an issue I had with making it work on non-TQ installs because at least on my system the cache folder is different than what your program was expecting. To make it work I replaced line 94 in cache.py with the following:
if servername == "tranquility": cachepath = os.path.join(path_buf.value, "CCP", "EVE", cacheFolderName, "cache") else: cachepath = os.path.join(path_buf.value, "CCP", "EVE", cacheFolderName + "_" + serverip, "cache")
I would guess that's because there's 2 ways to fire up sisi and they both produce a different folder (by name and by ip). I'll have to think of something clever to get around that :P _
Got Item? | EVE API? | Cache? |
Catari Taga
Centre Of Attention Rough Necks
|
Posted - 2010.04.02 14:21:00 -
[25]
Edited by: Catari Taga on 02/04/2010 14:25:00
Originally by: Entity I would guess that's because there's 2 ways to fire up sisi and they both produce a different folder (by name and by ip). I'll have to think of something clever to get around that :P
It's possible, in my case it's by IP and without /LUA:OFF. TQ is on E:\EVE. My two folders are thus:
%USERPROFILE%\Local Settings\Application Data\CCP\EVE\e_eve_tranquility %USERPROFILE%\Local Settings\Application Data\CCP\EVE\e_eve_singularity_87.237.38.50
Testing other combinations is unfortunately up to you - or just assume that whoever is using your library will probably be able solve the problem for their own system anyway. Thanks again for the library btw, it's been very useful.
|
Miss AmarrPriceCheck
|
Posted - 2010.05.08 17:59:00 -
[26]
This library is excellent and I look forward to its Tyrannis update. Thanks, Entity! |
Ix Forres
Caldari Vanguard Frontiers Intrepid Crossing
|
Posted - 2010.05.08 19:35:00 -
[27]
Originally by: Miss AmarrPriceCheck This library is excellent and I look forward to its Tyrannis update. Thanks, Entity!
Shouldn't need updating for Tyrannis, should work out of the box. -- Ix Forres EVE Application Developer EVE Metrics | accVIEW | I Tweet |
Miss AmarrPriceCheck
|
Posted - 2010.05.10 19:44:00 -
[28]
Originally by: Ix Forres
Originally by: Miss AmarrPriceCheck This library is excellent and I look forward to its Tyrannis update. Thanks, Entity!
Shouldn't need updating for Tyrannis, should work out of the box.
Oh, I was just going by what Entity said, above: "I may have to release an update soonÖ with the planetary interaction stuff coming in..."
If it continues to work fine on patch day, all the better. |
Entity
X-Factor Industries Synthetic Existence
|
Posted - 2010.05.10 20:46:00 -
[29]
Originally by: Miss AmarrPriceCheck
Originally by: Ix Forres
Originally by: Miss AmarrPriceCheck This library is excellent and I look forward to its Tyrannis update. Thanks, Entity!
Shouldn't need updating for Tyrannis, should work out of the box.
Oh, I was just going by what Entity said, above: "I may have to release an update soonÖ with the planetary interaction stuff coming in..."
If it continues to work fine on patch day, all the better.
It'll work. That was mostly referring to any possible new tables relating to PI, so I might have to add some table definitions. They can be manually accessed though, it's just less convenient.
_
Got Item? | EVE API? | Cache? |
cinderbrood
Caldari Delta Technologies And Development
|
Posted - 2010.05.15 04:37:00 -
[30]
Edited by: cinderbrood on 15/05/2010 04:38:11 Will we ever see a pure python version of this without the c/c++ extensions?
(I want to run this on IronPython) but alas they lack .pyd support.
Ie will we ever see a pure python version of your blue.pyd?
would open up cache support to the .net world as well. (albeit in a hacky manner)
Will Code For Isk. |
|
|
|
|
Pages: [1] 2 3 4 5 6 7 :: one page |
First page | Previous page | Next page | Last page |