This is in preparation for adding a new deob module which invokes the
bytecode deobfuscator, decompiler and AST deobfuscator in turn.
Signed-off-by: Graham <gpe@openrs2.org>
This already caught some cases of public members that should have been
private and one case where the inferred type was too specific.
Signed-off-by: Graham <gpe@openrs2.dev>
This allows two different classes in different modules to be refactored
to the same name - for example, Node in client and unpackclass.
Under the hood, it is implemented by prefixing each class name with the
library name and an exclamation mark (which is highly unlikely to appear
in a class name, as it is invalid Java syntax).
At first, prefixing class names with the library name feels like a bit
of a hack. However, it is much simpler than trying to track libraries
throughout the existing code. Furthermore, it allows us to avoid
forking ASM classes like Remapper.
The Fernflower driver was also changed to deobfuscate each library in
its own context, rather than trying to decompile them all in one go - by
the time classes reach Fernflower, the prefixes have already been
removed and Fernflower can't deal with duplicate class names either.
Signed-off-by: Graham <gpe@openrs2.dev>
At the moment, this means hard-coding the HD client but we should
reintroduce support for the SD and unsigned clients in the future.
This change provides a few benefits:
* Performance (~3x faster), as I generally only care about the HD
client.
* Allows us to continue injecting the NameMap for the HD client without
worrying about how to avoid injecting it (or support a different
NameMap) for the SD and unsigned clients.
Signed-off-by: Graham <gpe@openrs2.dev>
While the jaggl jar isn't actually obfuscated, this change means we'll
support the @OriginalXXX annotations (which is useful for local
variables, whose names aren't retained). The unused method, visibility
and final transformers will also tidy up the code slightly.
The new class/member filtering infrastructure is used to retain the
names of every class, method and field.
Signed-off-by: Graham <gpe@openrs2.dev>
The new system will make it easier to port the deobfuscator to different
revisions.
There are two main changes:
- The addition of a Profile class, which contains a list of excluded
classes, methods and fields, and the maximum obfuscated name length.
It is passed to Transformers that require it with dependency
injection.
- New ClassFilter and MemberFilter infrastructure. The MemberFilter
class adds support for filtering fields and methods based on the
owner and descriptor, in addition to the name. This makes the filters
more precise than the previous system. It also supports globs, which
makes it easy to filter whole groups of classes, fields and methods
in one go.
The Profile class uses a ClassFilter and MemberFilters to
represent the list of excluded classes, methods and fields.
A separate benefit is the addition of a separate entry points filter
to the Profile class. Prior to this commit, many Transformers re-used
the excluded method filter to find entry points, which is less precise
(many of the excluded methods in 550 are not entry points).
Support for filtering methods by owner and descriptor in addition to
name allows the DEFAULT_PUBLIC_CTOR_CLASSES Set in VisibilityTransformer to
be combined with the entry points filter.
In the future it might be desirable to split the excluded method set
into three separate sets:
- One to represent methods that can't be renamed.
- One to represent methods whose signature can't be changed.
- One to represent methods that can't be removed.
Signed-off-by: Graham <gpe@openrs2.dev>
This will provide a few benefits:
- Some of the implementations can now be turned into objects, reducing
memory allocation.
- A single Resource.compressLibrary() method will be able to take a
LibraryWriter, reducing duplication.
Signed-off-by: Graham <gpe@openrs2.dev>
Some of these method names only make sense when accompanied with the
class name. I think most of these were mistakes in the Java -> Kotlin
port.
Signed-off-by: Graham <gpe@openrs2.dev>
Fixes#99, which was caused by a method call in an unused method with a
non-constant dummy argument. This prevented the argument being removed,
but it wasn't obvious why as the method preventing the argument removal
was also removed.
The new transformer uses a different approach to the old one. It starts
exploring the call graph from the entry points, recursively analysing
method calls. Methods are only re-analysed if their possible argument
values change, with the Unknown value being used if we can't identify a
single integer constant at a call site. This prevents us from recursing
infinitely if the client code does.
While this first pass does simplify branches in order to ignore dummy
method calls that are never evaluated at runtime, it operates on a copy
of the method (as we initially ignore more calls while the argument
value sets are smaller, ignoring fewer calls as they build up).
A separate second pass simplifies branches on the original method and
inlines singleton constants, paving the way for the UnusedArgTransformer
to actually remove the newly unused arguments.
This new approach has several benefits:
- It is much faster than the old approach, as we only re-analyse methods
as required by argument value changes, rather than re-analysing every
method during every pass.
- It doesn't require special cases for dealing with mutually recursive
dummy calls. The old approach hard-coded special cases for mutually
recursive calls involving groups of 1 and 2 methods. The code for this
wasn't clean. Furthermore, while it was just about good enough for the
HD client, the SD client contains a mutually recursive group of 3
methods. The new approach is capable of dealing with mutually
recursive groups of any size.
Finally, the new transformer has a much cleaner implementation.
Signed-off-by: Graham <gpe@openrs2.dev>
The non-HD client appears to have been compiled with javac 1.1 or 1.2,
which compiles synchronized blocks differently to modern versions of
Java. For example, it uses a single try/catch block (modern versions use
two, the latter of which is recursive), subroutines (modern versions do
not) and the range of the try block is narrower.
Fernflower doesn't understand this old style of bytecode, and produces
empty synchronized blocks that do not cover the correct range.
The MONITORENTER sequence is also slightly different: it uses
ASTORE ALOAD MONITORENTER after pushing the monitor variable, rather
than DUP ASTORE MONITORENTER. This doesn't break Fernflower, but does
make it introduce a pointless assignment to a variable only used in the
synchronized() statement, rather than inlining the expression.
This commit introduces a transformer which fixes up Java 1.1-style
synchronized blocks just enough for Fernflower to be able to decompile
them cleanly. (It doesn't add a recursive try/catch, as Fernflower
ignores them.)
Signed-off-by: Graham <gpe@openrs2.dev>
game_unpacker.dat is the name of the file when downloaded in the
.jagex_cache_32/runescape directory. However, the other files containing
code are all named after the files on the web/JAGGRAB server.
This commit uses unpackclass.pack for consistency with all the other
files.
Signed-off-by: Graham <gpe@openrs2.dev>
This ensures the bundler/deobfuscator can only see classes that are part
of the Java runtime, and not classes from the bundler/deobfuscator
themselves.
Signed-off-by: Graham <gpe@openrs2.dev>
The split is no longer required - we haven't run javah automatically
since splitting the natives out into a separate repository.
Signed-off-by: Graham <gpe@openrs2.dev>
Removes GOTO instructions that point to the next instruction, typically
left over from stripping exceptions inserted by the obfuscator.
Although modern decompilers can handle such GOTOs fine, removing them
makes for a nicer IR representation.
Signed-off-by: Major <major@emulate.rs>
The transformer does _not_ check for use via reflection. The only cases
in the 550 and OSRS clients where methods are accessed via reflection
are either 1) JRE classes, 2) when the method name is sent from the
server.
PSVM and methods declared in TypedRemapper.EXCLUDED_METHODS are never
removed.
Signed-off-by: Major <major@emulate.rs>
Fernflower fails to decompile any exception handler with an end_pc
(`to` in Fernflower nomenclature) equal to the length of the code array,
even though this is permitted in the class file spec. This transformer
inserts a NOP at the end of any code array that has such an exception
handler.
Signed-off-by: Major <major@emulate.rs>
This allows us to avoid needing to worry about manipulating stack frames
in individual transformers, which adds lots of complexity. It's much
easier to just make ASM generate them for us.
Signed-off-by: Graham <gpe@openrs2.dev>
Fernflower was previously placing constructors in the middle of instance
methods. They look much nicer at the top.
This commit also ensures static methods appear before instance methods.
The client scrambles static fields and methods - moving each to a
random class where it does not logically belong.
This transformer moves all static methods to a new set of empty classes.
The intention is that the human refactoring the deobfuscated client can
move the static methods around with an IDE to classes where it makes
sense.
In theory we could use heuristics to move some methods around - e.g.
ideas I have include:
* If the method takes no arguments, move it to the same class as its
return type.
* If it's a void method that takes one argument, move it to the same
class as its single argument.
* Group related methods together based on where they are called.
However, some of the heuristics are probably fairly complicated to
implement. Furthermore, as they are heuristics, they will make mistakes.
In the naive approach, the Static* classes generated by this transformer
effectively act as a queue of methods that a human has yet to move to an
appropriate class. If the Static* classes are ever emptied after
refactoring, then we will know that all static methods have been moved
to an appropriate class.
We wouldn't be able to guarantee this if some methods were moved to
Class* classes with heuristics. The heuristics would also be fairly
complicated to implement. Therefore, the transformer does not use
heuristics.
Field scrambling support will be included in a future commit.
The obfuscator (or compiler, potentially?) converts INVOKEVIRTUAL
instructions to INVOKESPECIAL in the following circumstances:
* The owner of the method is the same as the class containing the
INVOKE instruction.
* The owner's ACC_FINAL flag is set.
When those two conditions are met, and when ACC_SUPER is set on the
owner (which is always true in the RuneScape client, and the flag is
ignored in modern JVMs), then INVOKESPECIAL and INVOKEVIRTUAL are
equivalent.
This has not caused problems until now. However, the future static
scrambling transformer will break the first condition in some cases, as
it moves methods between classes.
This transformer reverses the obfuscation, such that INVOKEVIRTUAL is
used again. INVOKEVIRTUAL instructions may be moved between classes
without complication.
In Java 1.4 and earlier, the compiler creates a synthetic method for
invoking Class.forName() and a static field for caching the Class<?>
instance.
This will not interact well with the static scrambling transformer. The
obfuscator strips some synthetic flags. Furthermore, the field and
method need to be moved together with the code that uses them for
decompilers like Fernflower to recognise them and convert them back to
class literals.
If Fernflower misses one, the decompiled Class.forName() helper does not
compile cleanly (as the bytecode is missing a CHECKCAST).
Rather than complicating the future static scrambling transformer, it's
easier to convert these to use LDC and upgrade the .class file version
to Java 5.
