Add initial implementation of the RS framing layer

I have spent a while thinking about the best way to implement this,
especially after I discovered that OSRS treats the packets sent during
the handshake stage as normal packets (see share/doc/protocol/login.md
for a longer explanation).

I have settled on a design where the same Rs2Decoder and Rs2Encoder
instances, which deal with decoding packet opcodes and lengths, will
remain the same for the entire connection - unless it switches into a
mode where a completely different framing scheme is used, such as
JAGGRAB and JS5.

Rs2Decoder and Rs2Encoder both support optional opcode encryption. The
initial cipher is set to NopStreamCipher. It will be set to an
IsaacRandom instance during the login process.

As the scrambled opcodes used in-game overlap with the opcodes used
during the login process, a class called Protocol is used to represent a
set of opcodes that can be used in a particular state. The protocol can
be changed at any time. Rs2Decoder sets isSingleDecode to true initially
to help do this safely, but once in the final in-game state this can be
disabled for performance.

Unlike previous servers, like Apollo and ScapeEmulator, intermediate
frame objects are not used. Rs2Decoder and Rs2Encoder call PacketCodec
to directly translate an inbound ByteBuf into a POJO, or an outbound
POJO into a ByteBuf. This saves a memory allocation per packet.

PacketCodec implementations must provide both an encode() and a decode()
method for every packet. This will ensure the library is usable as both
a client and server, allowing the development of tools, such as a JS5
client, a game server stress-tester or even a proxy server for
inspecting the flow of packets.

PacketCodec implementations may optionally override getLength() to
return their exact length, to optimise the size of the ByteBuf passed to
the decode() method. (It is not necessary to override getLength() for
fixed packets, as the size is already known.)

The 256-element packet length array is automatically generated by the
Protocol class based on the length attribute of the PacketCodec objects
passed to it. This is less error-prone than populating the array
manually. However, it does mean stub implementations of every packet
packet will need to be provided during development - all lengths must be
known in advance to ensure the client and server remain in sync.

For the moment, it is legal to throw NotImplementError from a decode()
method. Rs2Decoder catches this exception, prints a warning and then
proceed to the next packet. Once all packets have been implemented, this
code may be removed.

Signed-off-by: Graham <gpe@openrs2.org>
bzip2
Graham 4 years ago
parent 61d525c542
commit 95d6583dfe
  1. 1
      protocol/build.gradle.kts
  2. 16
      protocol/src/main/kotlin/org/openrs2/protocol/EmptyPacketCodec.kt
  3. 3
      protocol/src/main/kotlin/org/openrs2/protocol/Packet.kt
  4. 47
      protocol/src/main/kotlin/org/openrs2/protocol/PacketCodec.kt
  5. 6
      protocol/src/main/kotlin/org/openrs2/protocol/PacketLength.kt
  6. 22
      protocol/src/main/kotlin/org/openrs2/protocol/Protocol.kt
  7. 80
      protocol/src/main/kotlin/org/openrs2/protocol/Rs2Decoder.kt
  8. 60
      protocol/src/main/kotlin/org/openrs2/protocol/Rs2Encoder.kt
  9. 3
      protocol/src/test/kotlin/org/openrs2/protocol/EmptyPacket.kt
  10. 3
      protocol/src/test/kotlin/org/openrs2/protocol/FixedPacket.kt
  11. 18
      protocol/src/test/kotlin/org/openrs2/protocol/FixedPacketCodec.kt
  12. 18
      protocol/src/test/kotlin/org/openrs2/protocol/LengthMismatchPacketCodec.kt
  13. 103
      protocol/src/test/kotlin/org/openrs2/protocol/Rs2DecoderTest.kt
  14. 127
      protocol/src/test/kotlin/org/openrs2/protocol/Rs2EncoderTest.kt
  15. 6
      protocol/src/test/kotlin/org/openrs2/protocol/TestEmptyPacketCodec.kt
  16. 9
      protocol/src/test/kotlin/org/openrs2/protocol/TestStreamCipher.kt
  17. 3
      protocol/src/test/kotlin/org/openrs2/protocol/VariableByteOptimisedPacket.kt
  18. 23
      protocol/src/test/kotlin/org/openrs2/protocol/VariableByteOptimisedPacketCodec.kt
  19. 18
      protocol/src/test/kotlin/org/openrs2/protocol/VariableBytePacket.kt
  20. 19
      protocol/src/test/kotlin/org/openrs2/protocol/VariableBytePacketCodec.kt
  21. 3
      protocol/src/test/kotlin/org/openrs2/protocol/VariableShortOptimisedPacket.kt
  22. 23
      protocol/src/test/kotlin/org/openrs2/protocol/VariableShortOptimisedPacketCodec.kt
  23. 18
      protocol/src/test/kotlin/org/openrs2/protocol/VariableShortPacket.kt
  24. 19
      protocol/src/test/kotlin/org/openrs2/protocol/VariableShortPacketCodec.kt

@ -4,6 +4,7 @@ plugins {
}
dependencies {
api(project(":crypto"))
api("io.netty:netty-codec:${Versions.netty}")
implementation(project(":buffer"))

@ -0,0 +1,16 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
public abstract class EmptyPacketCodec<T : Packet>(
private val packet: T,
opcode: Int
) : PacketCodec<T>(packet.javaClass, opcode, length = 0) {
override fun decode(input: ByteBuf): T {
return packet
}
override fun encode(input: T, output: ByteBuf) {
// empty
}
}

@ -0,0 +1,3 @@
package org.openrs2.protocol
public interface Packet

@ -0,0 +1,47 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
import io.netty.buffer.ByteBufAllocator
public abstract class PacketCodec<T : Packet>(
public val type: Class<T>,
public val opcode: Int,
public val length: Int
) {
init {
require(opcode in 0 until 256)
require(length >= PacketLength.VARIABLE_SHORT)
}
public abstract fun decode(input: ByteBuf): T
public abstract fun encode(input: T, output: ByteBuf)
public open fun getLength(input: T): Int {
return length
}
public fun allocateBuffer(alloc: ByteBufAllocator, input: T, preferDirect: Boolean): ByteBuf {
val payloadLen = getLength(input)
if (payloadLen < 0) {
return if (preferDirect) {
alloc.ioBuffer()
} else {
alloc.heapBuffer()
}
}
val headerLen = when (length) {
PacketLength.VARIABLE_BYTE -> 2
PacketLength.VARIABLE_SHORT -> 3
else -> 1
}
val totalLen = headerLen + payloadLen
return if (preferDirect) {
alloc.ioBuffer(totalLen, totalLen)
} else {
alloc.heapBuffer(totalLen, totalLen)
}
}
}

@ -0,0 +1,6 @@
package org.openrs2.protocol
public object PacketLength {
public const val VARIABLE_SHORT: Int = -2
public const val VARIABLE_BYTE: Int = -1
}

@ -0,0 +1,22 @@
package org.openrs2.protocol
public class Protocol(vararg codecs: PacketCodec<*>) {
private val decoders = arrayOfNulls<PacketCodec<*>>(256)
private val encoders = codecs.associateBy(PacketCodec<*>::type)
init {
for (codec in codecs) {
decoders[codec.opcode] = codec
}
}
public fun getDecoder(opcode: Int): PacketCodec<*>? {
require(opcode in decoders.indices)
return decoders[opcode]
}
@Suppress("UNCHECKED_CAST")
public fun <T : Packet> getEncoder(type: Class<T>): PacketCodec<T>? {
return encoders[type] as PacketCodec<T>?
}
}

@ -0,0 +1,80 @@
package org.openrs2.protocol
import com.github.michaelbull.logging.InlineLogger
import io.netty.buffer.ByteBuf
import io.netty.channel.ChannelHandlerContext
import io.netty.handler.codec.ByteToMessageDecoder
import io.netty.handler.codec.DecoderException
import org.openrs2.crypto.NopStreamCipher
import org.openrs2.crypto.StreamCipher
public class Rs2Decoder(public var protocol: Protocol) : ByteToMessageDecoder() {
private enum class State {
READ_OPCODE,
READ_LENGTH,
READ_PAYLOAD
}
public var cipher: StreamCipher = NopStreamCipher
private var state = State.READ_OPCODE
private lateinit var decoder: PacketCodec<*>
private var length = 0
init {
isSingleDecode = true
}
override fun decode(ctx: ChannelHandlerContext, input: ByteBuf, out: MutableList<Any>) {
if (state == State.READ_OPCODE) {
if (!input.isReadable) {
return
}
val opcode = (input.readUnsignedByte().toInt() - cipher.nextInt()) and 0xFF
decoder = protocol.getDecoder(opcode) ?: throw DecoderException("Unsupported opcode: $opcode")
length = decoder.length
state = State.READ_LENGTH
}
if (state == State.READ_LENGTH) {
when (length) {
PacketLength.VARIABLE_BYTE -> {
if (!input.isReadable) {
return
}
length = input.readUnsignedByte().toInt()
}
PacketLength.VARIABLE_SHORT -> {
if (input.readableBytes() < 2) {
return
}
length = input.readUnsignedShort()
}
}
state = State.READ_PAYLOAD
}
if (state == State.READ_PAYLOAD) {
if (input.readableBytes() < length) {
return
}
out += try {
decoder.decode(input.readSlice(length))
} catch (ex: NotImplementedError) {
// TODO(gpe): remove this catch block when every packet is implemented
logger.warn { "Skipping unimplemented packet: ${decoder.javaClass}" }
}
state = State.READ_OPCODE
}
}
private companion object {
private val logger = InlineLogger()
}
}

@ -0,0 +1,60 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
import io.netty.channel.ChannelHandlerContext
import io.netty.handler.codec.EncoderException
import io.netty.handler.codec.MessageToByteEncoder
import org.openrs2.crypto.NopStreamCipher
import org.openrs2.crypto.StreamCipher
public class Rs2Encoder(public var protocol: Protocol) : MessageToByteEncoder<Packet>(Packet::class.java) {
public var cipher: StreamCipher = NopStreamCipher
override fun encode(ctx: ChannelHandlerContext, msg: Packet, out: ByteBuf) {
val encoder = protocol.getEncoder(msg.javaClass)
?: throw EncoderException("Unsupported packet type: ${msg.javaClass}")
out.writeByte(encoder.opcode + cipher.nextInt())
val len = encoder.length
val lenIndex = out.writerIndex()
when (len) {
PacketLength.VARIABLE_BYTE -> out.writeZero(1)
PacketLength.VARIABLE_SHORT -> out.writeZero(2)
}
val payloadIndex = out.writerIndex()
encoder.encode(msg, out)
val written = out.writerIndex() - payloadIndex
when (len) {
PacketLength.VARIABLE_BYTE -> {
if (written >= 256) {
throw EncoderException("Variable byte payload too long: $written bytes")
}
out.setByte(lenIndex, written)
}
PacketLength.VARIABLE_SHORT -> {
if (written >= 65536) {
throw EncoderException("Variable short payload too long: $written bytes")
}
out.setShort(lenIndex, written)
}
else -> {
if (written != len) {
throw EncoderException("Fixed payload length mismatch (expected $len bytes, got $written bytes)")
}
}
}
}
override fun allocateBuffer(ctx: ChannelHandlerContext, msg: Packet, preferDirect: Boolean): ByteBuf {
val encoder = protocol.getEncoder(msg.javaClass)
?: throw EncoderException("Unsupported packet type: ${msg.javaClass}")
return encoder.allocateBuffer(ctx.alloc(), msg, preferDirect)
}
}

@ -0,0 +1,3 @@
package org.openrs2.protocol
object EmptyPacket : Packet

@ -0,0 +1,3 @@
package org.openrs2.protocol
internal data class FixedPacket(val value: Int) : Packet

@ -0,0 +1,18 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object FixedPacketCodec : PacketCodec<FixedPacket>(
type = FixedPacket::class.java,
opcode = 0,
length = 4
) {
override fun decode(input: ByteBuf): FixedPacket {
val value = input.readInt()
return FixedPacket(value)
}
override fun encode(input: FixedPacket, output: ByteBuf) {
output.writeInt(input.value)
}
}

@ -0,0 +1,18 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object LengthMismatchPacketCodec : PacketCodec<FixedPacket>(
type = FixedPacket::class.java,
opcode = 0,
length = 5
) {
override fun decode(input: ByteBuf): FixedPacket {
val value = input.readInt()
return FixedPacket(value)
}
override fun encode(input: FixedPacket, output: ByteBuf) {
output.writeInt(input.value)
}
}

@ -0,0 +1,103 @@
package org.openrs2.protocol
import io.netty.buffer.Unpooled
import io.netty.channel.embedded.EmbeddedChannel
import io.netty.handler.codec.DecoderException
import org.junit.jupiter.api.assertThrows
import org.openrs2.buffer.wrappedBuffer
import kotlin.test.Test
import kotlin.test.assertEquals
object Rs2DecoderTest {
@Test
fun testDecode() {
testDecode(byteArrayOf(0, 0x11, 0x22, 0x33, 0x44), FixedPacket(0x11223344))
testDecode(byteArrayOf(1, 3, 0x11, 0x22, 0x33), VariableBytePacket(byteArrayOf(0x11, 0x22, 0x33)))
testDecode(byteArrayOf(2, 0, 3, 0x11, 0x22, 0x33), VariableShortPacket(byteArrayOf(0x11, 0x22, 0x33)))
testDecode(byteArrayOf(5), EmptyPacket)
}
@Test
fun testFragmented() {
testFragmented(byteArrayOf(0, 0x11, 0x22, 0x33, 0x44), FixedPacket(0x11223344))
testFragmented(byteArrayOf(1, 3, 0x11, 0x22, 0x33), VariableBytePacket(byteArrayOf(0x11, 0x22, 0x33)))
testFragmented(byteArrayOf(2, 0, 3, 0x11, 0x22, 0x33), VariableShortPacket(byteArrayOf(0x11, 0x22, 0x33)))
}
@Test
fun testUnsupported() {
val channel = EmbeddedChannel(Rs2Decoder(Protocol()))
assertThrows<DecoderException> {
channel.writeInbound(wrappedBuffer(0))
}
}
@Test
fun testEncryptedOpcode() {
val decoder = Rs2Decoder(Protocol(FixedPacketCodec))
decoder.cipher = TestStreamCipher
val channel = EmbeddedChannel(decoder)
channel.writeInbound(wrappedBuffer(10, 0x11, 0x22, 0x33, 0x44))
val actual = channel.readInbound<Packet>()
assertEquals(FixedPacket(0x11223344), actual)
}
@Test
fun testSwitchProtocol() {
val decoder = Rs2Decoder(Protocol(FixedPacketCodec))
val channel = EmbeddedChannel(decoder)
channel.writeInbound(wrappedBuffer(0, 0x11, 0x22, 0x33, 0x44))
channel.readInbound<Packet>()
assertThrows<DecoderException> {
channel.writeInbound(wrappedBuffer(5))
}
decoder.protocol = Protocol(TestEmptyPacketCodec)
channel.writeInbound(wrappedBuffer(5))
val actual = channel.readInbound<Packet>()
assertEquals(EmptyPacket, actual)
assertThrows<DecoderException> {
channel.writeInbound(wrappedBuffer(0, 0x11, 0x22, 0x33, 0x44))
}
}
private fun testDecode(buf: ByteArray, expected: Packet) {
val channel = EmbeddedChannel(Rs2Decoder(Protocol(
FixedPacketCodec,
VariableBytePacketCodec,
VariableShortPacketCodec,
VariableByteOptimisedPacketCodec,
VariableShortOptimisedPacketCodec,
TestEmptyPacketCodec
)))
channel.writeInbound(Unpooled.wrappedBuffer(buf))
val actual = channel.readInbound<Packet>()
assertEquals(expected, actual)
}
private fun testFragmented(buf: ByteArray, expected: Packet) {
val channel = EmbeddedChannel(Rs2Decoder(Protocol(
FixedPacketCodec,
VariableBytePacketCodec,
VariableShortPacketCodec,
VariableByteOptimisedPacketCodec,
VariableShortOptimisedPacketCodec
)))
for (b in buf) {
channel.writeInbound(wrappedBuffer(b))
}
val actual = channel.readInbound<Packet>()
assertEquals(expected, actual)
}
}

@ -0,0 +1,127 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
import io.netty.buffer.Unpooled
import io.netty.channel.embedded.EmbeddedChannel
import io.netty.handler.codec.EncoderException
import org.junit.jupiter.api.assertThrows
import org.openrs2.buffer.use
import org.openrs2.buffer.wrappedBuffer
import kotlin.test.Test
import kotlin.test.assertEquals
object Rs2EncoderTest {
@Test
fun testEncode() {
testEncode(FixedPacket(0x11223344), byteArrayOf(0, 0x11, 0x22, 0x33, 0x44))
testEncode(VariableBytePacket(byteArrayOf(0x11, 0x22, 0x33)), byteArrayOf(1, 3, 0x11, 0x22, 0x33))
testEncode(VariableShortPacket(byteArrayOf(0x11, 0x22, 0x33)), byteArrayOf(2, 0, 3, 0x11, 0x22, 0x33))
testEncode(EmptyPacket, byteArrayOf(5))
}
@Test
fun testTooLong() {
val channel = EmbeddedChannel(Rs2Encoder(Protocol(
VariableBytePacketCodec,
VariableShortPacketCodec
)))
channel.writeOutbound(VariableShortPacket(ByteArray(255)))
channel.readOutbound<ByteBuf>().release()
channel.writeOutbound(VariableShortPacket(ByteArray(65535)))
channel.readOutbound<ByteBuf>().release()
assertThrows<EncoderException> {
channel.writeOutbound(VariableBytePacket(ByteArray(256)))
}
assertThrows<EncoderException> {
channel.writeOutbound(VariableShortPacket(ByteArray(65536)))
}
}
@Test
fun testUnsupported() {
val channel = EmbeddedChannel(Rs2Encoder(Protocol()))
assertThrows<EncoderException> {
channel.writeOutbound(FixedPacket(0x11223344))
}
}
@Test
fun testLengthMismatch() {
val channel = EmbeddedChannel(Rs2Encoder(Protocol(LengthMismatchPacketCodec)))
assertThrows<EncoderException> {
channel.writeOutbound(FixedPacket(0x11223344))
}
}
@Test
fun testLengthOptimised() {
testEncode(VariableByteOptimisedPacket(byteArrayOf(0x11, 0x22, 0x33)), byteArrayOf(3, 3, 0x11, 0x22, 0x33))
testEncode(VariableShortOptimisedPacket(byteArrayOf(0x11, 0x22, 0x33)), byteArrayOf(4, 0, 3, 0x11, 0x22, 0x33))
}
@Test
fun testEncryptedOpcode() {
val encoder = Rs2Encoder(Protocol(FixedPacketCodec))
encoder.cipher = TestStreamCipher
val channel = EmbeddedChannel(encoder)
channel.writeOutbound(FixedPacket(0x11223344))
channel.readOutbound<ByteBuf>().use { actual ->
wrappedBuffer(10, 0x11, 0x22, 0x33, 0x44).use { expected ->
assertEquals(expected, actual)
}
}
}
@Test
fun testSwitchProtocol() {
val encoder = Rs2Encoder(Protocol(FixedPacketCodec))
val channel = EmbeddedChannel(encoder)
channel.writeOutbound(FixedPacket(0x11223344))
channel.readOutbound<ByteBuf>().release()
assertThrows<EncoderException> {
channel.writeOutbound(EmptyPacket)
}
encoder.protocol = Protocol(TestEmptyPacketCodec)
channel.writeOutbound(EmptyPacket)
channel.readOutbound<ByteBuf>().use { actual ->
wrappedBuffer(5).use { expected ->
assertEquals(expected, actual)
}
}
assertThrows<EncoderException> {
channel.writeOutbound(FixedPacket(0x11223344))
}
}
private fun testEncode(packet: Packet, expected: ByteArray) {
val channel = EmbeddedChannel(Rs2Encoder(Protocol(
FixedPacketCodec,
VariableBytePacketCodec,
VariableShortPacketCodec,
VariableByteOptimisedPacketCodec,
VariableShortOptimisedPacketCodec,
TestEmptyPacketCodec
)))
channel.writeOutbound(packet)
channel.readOutbound<ByteBuf>().use { actual ->
Unpooled.wrappedBuffer(expected).use { expected ->
assertEquals(expected, actual)
}
}
}
}

@ -0,0 +1,6 @@
package org.openrs2.protocol
object TestEmptyPacketCodec : EmptyPacketCodec<EmptyPacket>(
packet = EmptyPacket,
opcode = 5
)

@ -0,0 +1,9 @@
package org.openrs2.protocol
import org.openrs2.crypto.StreamCipher
object TestStreamCipher : StreamCipher {
override fun nextInt(): Int {
return 10
}
}

@ -0,0 +1,3 @@
package org.openrs2.protocol
internal class VariableByteOptimisedPacket(val value: ByteArray) : Packet

@ -0,0 +1,23 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object VariableByteOptimisedPacketCodec : PacketCodec<VariableByteOptimisedPacket>(
type = VariableByteOptimisedPacket::class.java,
opcode = 3,
length = PacketLength.VARIABLE_BYTE
) {
override fun decode(input: ByteBuf): VariableByteOptimisedPacket {
val value = ByteArray(input.readableBytes())
input.readBytes(value)
return VariableByteOptimisedPacket(value)
}
override fun encode(input: VariableByteOptimisedPacket, output: ByteBuf) {
output.writeBytes(input.value)
}
override fun getLength(input: VariableByteOptimisedPacket): Int {
return input.value.size
}
}

@ -0,0 +1,18 @@
package org.openrs2.protocol
internal class VariableBytePacket(val value: ByteArray) : Packet {
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (javaClass != other?.javaClass) return false
other as VariableBytePacket
if (!value.contentEquals(other.value)) return false
return true
}
override fun hashCode(): Int {
return value.contentHashCode()
}
}

@ -0,0 +1,19 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object VariableBytePacketCodec : PacketCodec<VariableBytePacket>(
type = VariableBytePacket::class.java,
opcode = 1,
length = PacketLength.VARIABLE_BYTE
) {
override fun decode(input: ByteBuf): VariableBytePacket {
val value = ByteArray(input.readableBytes())
input.readBytes(value)
return VariableBytePacket(value)
}
override fun encode(input: VariableBytePacket, output: ByteBuf) {
output.writeBytes(input.value)
}
}

@ -0,0 +1,3 @@
package org.openrs2.protocol
internal class VariableShortOptimisedPacket(val value: ByteArray) : Packet

@ -0,0 +1,23 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object VariableShortOptimisedPacketCodec : PacketCodec<VariableShortOptimisedPacket>(
type = VariableShortOptimisedPacket::class.java,
opcode = 4,
length = PacketLength.VARIABLE_SHORT
) {
override fun decode(input: ByteBuf): VariableShortOptimisedPacket {
val value = ByteArray(input.readableBytes())
input.readBytes(value)
return VariableShortOptimisedPacket(value)
}
override fun encode(input: VariableShortOptimisedPacket, output: ByteBuf) {
output.writeBytes(input.value)
}
override fun getLength(input: VariableShortOptimisedPacket): Int {
return input.value.size
}
}

@ -0,0 +1,18 @@
package org.openrs2.protocol
internal class VariableShortPacket(val value: ByteArray) : Packet {
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (javaClass != other?.javaClass) return false
other as VariableShortPacket
if (!value.contentEquals(other.value)) return false
return true
}
override fun hashCode(): Int {
return value.contentHashCode()
}
}

@ -0,0 +1,19 @@
package org.openrs2.protocol
import io.netty.buffer.ByteBuf
internal object VariableShortPacketCodec : PacketCodec<VariableShortPacket>(
type = VariableShortPacket::class.java,
opcode = 2,
length = PacketLength.VARIABLE_SHORT
) {
override fun decode(input: ByteBuf): VariableShortPacket {
val value = ByteArray(input.readableBytes())
input.readBytes(value)
return VariableShortPacket(value)
}
override fun encode(input: VariableShortPacket, output: ByteBuf) {
output.writeBytes(input.value)
}
}
Loading…
Cancel
Save