Graham
4337020b6b
Add endpoint for exporting all keys in bulk
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
93ee863e20
Add initial support for plotting a map of valid/invalid XTEA keys
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
dbd169aa1a
Make fastutil part of the cache library's API
...
The group unpacking code exposes a fastutil type. (In the long run this
might be replaced with an array, allowing us to revert this commit or
drop the fastutil dependency entirely.)
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
da3cc57b3f
Add link to download keys in the text format
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f6961a50f1
Add cache size in bytes to master_index_stats
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
8bc25e7e47
Set compression lewel to -9
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
aad469c05d
Generate XTEA key zip reproducibly
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
c4f807d38f
Add X-Forwarded-For support
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
2d136c89e9
Add zipped text version of the XTEA key endpoint
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a02eb169f4
Fix NPE in TextKeyReader if the input is invalid
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a6fa7bf5b6
Throw exception before importing a cache if its master index is empty
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ae88bca924
Add ent command for analysing the randomness of keys
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
e4776f7c3c
Only call games.single() when game is accessed
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f870410a09
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
e6af0affb6
Ignore corrupt indexes in Js5MasterIndex::create
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
bf86fa48e9
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
8c395bcf4b
Add index on key_id
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a8fbfd851c
Replace PL/pgSQL functions with inlineable SQL functions
...
This speeds up the resolved_* views by a reasonable amount, though it
does mean we won't be able to use the smarter resolution logic (which is
far too slow anyway at the moment, so I'm not sure what I'm going to do
about that in the future...)
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
5d1fa42623
Remove pointless LATERAL JOINs
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
b792b65f5e
Remove smarter resolution algorithm for now
...
It's significantly slower. I'm not sure why yet.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
3a897d4e67
Downgrade Netty
...
The native libraries haven't been linked correctly in 4.1.60.Final.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
18d7fdc343
Resolve collisions by prioritising groups from the same source
...
This doesn't fix all cases, as a master index can have multiple sources,
each with a distinct copy of the same (archive, group, checksum,
version) tuple. However, it's probably as good as we'll be able to do
automatically - and it'll work particularly well for master indexes
downloaded directly over JS5, where we won't have done multiple imports
of the same cache.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
1f735f4b99
Remove duplicate descriptions
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f04d163d98
Add non-truncated version numbers to the collisions view
...
Will be useful for determining if a collision is caused by version
truncation.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
f59d9213eb
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
c49ce684c8
Optimize master_index_stats
...
We no longer need COUNT DISTINCT - the resolve_* functions can only
return a single row.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
24d5b91ebe
Mark functions as stable, strict and parallel safe
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
cba34b54c9
Add resolve_{group,index} functions
...
These functions reduce the amount of group resolution logic
significantly, concentrating it in a single place. In addition to the
usual code de-duplication benefits, many of the queries are now much
simpler as the complexity is hidden behind the function calls.
This change also allows us to make the group resolution logic more
complicated. The first change is that the functions are guaranteed to
only return a single row, which was not true with the old JOIN-based
approach. The row that is chosen is chosen deterministically.
The resolution logic will probably be improved in the future, so we can
make a better decision where there are multiple possible groups, due to
collisions.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
a8794aca99
Include empty master index in master_index_stats
...
This fixes the caches with an empty master index from displaying as
'Calculating...'.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
b685d901aa
Mark empty master index as invalid
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
38be1905cb
Fix cache source URL list
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
006c8f7b9b
Improve grammar
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
eeb82d2e47
Fix typo in master_index_stats JOIN
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
9bce5cc1b3
Replace LEFT JOIN on sources/games with a JOIN
...
Each master index now has at least one source.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4ad9b31aae
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
c0f9590f48
Add foreign key to source_groups
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4353d8a616
Refactor identikit-related code
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
69a511a3c5
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
b98d045cfe
Add group source tracking
...
There are a few collisions in the production archive. I suspect these
are due to poorly modified caches, and tracking the source(s) of each
group will make it easier to determine which cache is probably
problematic.
This change also has the benefit of removing a lot of the hacky source
name/description merging logic.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4856f79152
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
853f2cff91
Add assignment operator support to HexLiteralTransformer
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
99b725dc8f
Rename clearBuffer to fillBuffer
...
Unfortunately the dummy argument removal strips away most of the
arguments.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
538f4d06c6
Refactor (mostly) font-related code
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
4deabceb9e
Move container_id to the end of the groups index
...
I think this will speed up the collisions view.
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
d8fd55f5cc
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
0007a2917b
Update dependencies
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
bb293d3e3f
Add missing ORDER BY to queries fetching the next container/key
...
I have no idea how I missed this!
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
995cdbd1b7
Add index to speed up the XTEA key brute force command
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
ea63769fe9
Add container IDs to the collisions view for debugging
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
Graham
76dba3f404
Count indexes with a zero version/checksum as valid
...
Signed-off-by: Graham <gpe@openrs2.org>
4 years ago
