Wrapping a Native Library with Maven

I recently converted a large project to build with Maven. The project contained both C++ and Java code, and produced a web application, a standalone server application, plus a number of small command line tools. The project used a large number of open-source Java libraries, and Maven tamed these easily. The native C++ library proved harder, and this is the approach I took.

The code snippets below are part of a complete example that builds a tiny Java/C++ application under Linux. This should port easily to other Unix-like platforms, and may provide some help to performing the same task under Windows. The example is available in tar.gz and zip formats.

There were three relevant components: the native library, its JNI wrapper, and the Java application that used it.

Native Library

The native library already had a makefile based build process. I decided to leave this unchanged: the library has upstream developers who do not use Java, and any other approach would have required changing pom.xml every time I imported a new version. I also decided to create a single artifact containing both header files and a shared library, and to build for a single target platform. If you have a multi-platform build, you will need a more complex set of profiles to create an artifact for each platform.

The result uses the Maven Exec Plugin to execute make and the Maven Assembly Plugin to package the results into a tar file.

<plugin>
	<!-- Use exec plugin to call make, to build or clean library for Linux	-->
	<groupid>org.codehaus.mojo</groupid>
	<artifactid>exec-maven-plugin</artifactid>
	<version>1.1</version>
	<executions>
		<execution>
			<!-- make all in the Maven compile phase -->
			<id>buildlib</id>
			<phase>compile</phase>
			<goals><goal>exec</goal></goals>
			<configuration>
				<executable>make</executable>
				<arguments>
					<argument>all</argument>
				</arguments>
			</configuration>
		</execution>
		<execution>
			<!-- Make clean in the Maven clean phase -->
			<id>cleanlib</id>
			<phase>clean</phase>
			<goals><goal>exec</goal></goals>
			<configuration>
				<executable>make</executable>
				<arguments>
					<argument>clean</argument>
				</arguments>
			</configuration>
		</execution>
	</executions>
</plugin>

The build process for the library produces a single shared library, libexample.so. Code that uses this library will also need the library header file, example.h. The assembly descriptor builds a tar file, containing a lib and include directory:

<assembly>
	<!-- Dependent projects will use this id as the classifier to match this assembly -->
	<id>libs</id>
<formats>
<format>tar</format>
	</formats>
	<files>
		<file>
			<source>libexample.so</source>
			<outputdirectory>lib</outputdirectory>
			<filemode>0644</filemode>
		</file>
		<file>
			<source>example.h</source>
			<outputdirectory>include</outputdirectory>
			<filemode>0644</filemode>
		</file>
	</files>
</assembly>

JNI Wrapper

The JNI wrapper contains two sub-projects. One contains Java classes that provide the Java interface to the library, producing a JAR, and the other implements the native methods of these classes, producing a shared library in .so format. The native code project has a dependency on the library assembly, using the classifier of the dependency to match the id from the assembly descriptor.

<dependency>
	<groupid>${parent.groupId}</groupid>
	<artifactid>library</artifactid>
	<version>${project.version}</version>
	<!-- Reference the ID of the assembly -->
	<classifier>libs</classifier>
	<type>tar</type>
</dependency>

Before building the native code, the library dependency must be unpacked using the Maven Dependency Plugin. The assembly plugin creates a tar file with an initial path based on the project name and version, which can be unpacked in the base of the build directory:

<plugin>
	<groupid>org.apache.maven.plugins</groupid>
	<artifactid>maven-dependency-plugin</artifactid>
	<executions>
	<!-- Unpack header files and libraries for build -->
		<execution>
			<id>unpack</id>
			<phase>generate-sources</phase>
			<goals>
				<goal>unpack-dependencies</goal>
			</goals>
			<configuration>
				<includetypes>tar</includetypes>
				<outputdirectory>${project.build.directory}</outputdirectory>
			</configuration>
		</execution>
	</executions>
</plugin>

The Maven Native Plugin generates the JNI header files, and controls the compilation of the native code. Our library include files go on to the source directory list:

<source>
	<directory>
		${project.build.directory}/library-${project.version}/include
	</directory>
</source>

Our library, libexample.so, must be listed in the linker options. The options also explicitly reference libstdc++, as the library source is compiled in C++.

	<linkerstartoptions>
	<linkerstartoption>
		-shared
		-L${project.build.directory}/library-${project.version}/lib
		-lstdc++
		-lexample
	</linkerstartoption>
</linkerstartoptions>

Application

The application contains Java code and a small shell script to execute the assembly. The application project unpacks the library tar file in exactly the same way as the native wrapper, but this is only required to repack the library file into the final assembly.

The assembly contains all dependent JAR files (including the local application code):

<dependencyset>
	<includes>
		<include>*:jar</include>
	</includes>
	<outputdirectory>lib</outputdirectory>
</dependencyset>

It enforces a fixed file name for the native wrapper code, to ensure that the loadLibrary call in the Java code finds it:

<dependencyset>
	<includes>
		<include>uk.co.humboldt.MavenJNIExample:WrapperNative</include>
	</includes>
	<!-- Rename to a fixed name to match the loading code
	     System.loadLibrary("wrapper"); -->
	<outputfilenamemapping>
	libwrapper.so
	</outputfilenamemapping>
	<outputdirectory>lib</outputdirectory>
</dependencyset>

The assembly also contains the native shared library:

<fileset>
	<directory>
		${project.build.directory}/library-${project.version}/lib
	</directory>
	<outputdirectory>lib</outputdirectory>
	<includes>
		<include>*.so</include>
	</includes>
	<usedefaultexcludes>true</usedefaultexcludes>
	<filemode>0644</filemode>
	<directorymode>0755</directorymode>
</fileset>

The end result is a lib directory which contains the following files:

  • Application-1.0-SNAPSHOT.jar The application code.
  • WrapperJava-1.0-SNAPSHOT.jar The Java code that wraps the native library.
  • libwrapper.so The native functions of the wrapper classes.
  • libexample.so The original native library.

The final component is the shell script, which adds the lib directory to both the Java classpath and LD_LIBRARY_PATH.

#!/bin/sh
installdir=`dirname $0`
export LD_LIBRARY_PATH=$installdir/lib:$LD_LIBRARY_PATH
java -classpath $installdir/lib/* uk.co.humboldt.MavenJNIExample.Application $*

Notes

The complete example is available can be downloaded in either tar.gz or zip formats.

Building the entire project with mvn compile will not work, as it will not build the intermediate assemblies. Either use mvn package from the top level, or install the assemblies into your local repository.

Users of Maven Integration for Eclipse will need to use an external Maven implementation, as the native plugin fails using the embedded Maven.