Solving unreliable wifi connections on an Intel AC-8260 in Ubuntu

Published / by maurice / Leave a Comment

The problem

A few months ago, I started having intermittent wifi connection problems. It happened around the same time I upgraded the Ubuntu installation to 17.10, so I immediately suspected that something was updated to a version that didn’t work for my wifi hardware.

Quite some serious Googling later – it’s incredible how many people have problems with their wireless connections on Linux – I came across a viable solution. Pretty silly in its simplicity, but it works…

First off, as the title says, I have an Intel Dual Band Wireless-AC 8260 card in my laptop. This can be verified by:

$ lspci -nnk | grep 0280 -A2
70:00.0 Network controller [0280]: Intel Corporation Wireless 8260 [8086:24f3] (rev 3a)
  Subsystem: Intel Corporation Dual Band Wireless-AC 8260 [8086:0010]
  Kernel driver in use: iwlwifi

The cause

I assumed the problem was caused by the version of the driver, iwlwifi, as this was suggested in a number of the pages I found. Other options included problems with the 820.11N-layer, which can be disabled, but that was not necessary in my case.

To see what version is loaded:

$ dmesg | grep iwlwifi
[    8.598701] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-33.ucode failed with error -2
[    8.599260] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-32.ucode failed with error -2
[    8.614217] iwlwifi 0000:70:00.0: loaded firmware version 31.560484.0 op_mode iwlmvm
[    8.659485] iwlwifi 0000:70:00.0: Detected Intel(R) Dual Band Wireless AC 8260, REV=0x208
[    8.736615] iwlwifi 0000:70:00.0: base HW address: a4:34:d9:7f:95:c2
[    8.811521] ieee80211 phy0: Selected rate control algorithm 'iwl-mvm-rs'
[    8.813223] iwlwifi 0000:70:00.0 wlp112s0: renamed from wlan0

So we see that several versions are tried (33 and 32) before version 31 is actually loaded.

We can see how this is appropriate for my setup by looking at the corresponding module info (which indicates the highest version to be attempted; after each failure it decreases the number by one and attempts again):

$ sudo modinfo iwlwifi | grep iwlwifi-8000C
firmware:       iwlwifi-8000C-33.ucode

And taking into account the available drivers on the system:

$ ll /lib/firmware/iwlwifi-8000C*
-rw-r--r-- 1 root root 1745176 mrt 30  2017 /lib/firmware/iwlwifi-8000C-13.ucode
-rw-r--r-- 1 root root 2351636 mrt 30  2017 /lib/firmware/iwlwifi-8000C-16.ucode
-rw-r--r-- 1 root root 2394060 nov 17 17:40 /lib/firmware/iwlwifi-8000C-21.ucode
-rw-r--r-- 1 root root 2227284 dec  5 21:31 /lib/firmware/iwlwifi-8000C-27.ucode
-rw-r--r-- 1 root root 2310116 dec  6 15:43 /lib/firmware/iwlwifi-8000C-31.ucode

The premise is that this version 31 was packaged with Ubuntu 17.10 (Artful) and not yet in 17.04 (Zesty), which can be verified by looking at the difference between these two pages:

Indeed, in Artful versions 27 and 31 were added (not all consecutive version numbers are applicable to all card types, as can be seen in the table on this page: https://wireless.wiki.kernel.org/en/users/drivers/iwlwifi/core_release).

The solution

As said above, the solution to my problem proved very simple: downgrade the driver to a version that is available and works. To accomplish that, just rename the driver file in /lib/firmware to something else, so that it isn’t loaded anymore at startup (e.g. give it a .unwanted suffix).

After rebooting*, that looks something like this:

$ dmesg | grep iwl
[    8.931942] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-33.ucode failed with error -2
[    8.932340] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-32.ucode failed with error -2
[    8.934736] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-31.ucode failed with error -2
[    8.935383] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-30.ucode failed with error -2
[    8.935830] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-29.ucode failed with error -2
[    8.935842] iwlwifi 0000:70:00.0: Direct firmware load for iwlwifi-8000C-28.ucode failed with error -2
[    8.940888] iwlwifi 0000:70:00.0: loaded firmware version 27.541033.0 op_mode iwlmvm
[    8.959754] iwlwifi 0000:70:00.0: Detected Intel(R) Dual Band Wireless AC 8260, REV=0x208
[    9.036883] iwlwifi 0000:70:00.0: base HW address: a4:34:d9:7f:95:c2
[    9.112879] ieee80211 phy0: Selected rate control algorithm 'iwl-mvm-rs'
[    9.114525] iwlwifi 0000:70:00.0 wlp112s0: renamed from wlan0

As you can see, version 31 is no longer loaded, and the next highest available version 27 is loaded instead.

For me, that did the trick. If it hadn’t, I could have tried another step down, to version 21, which has worked fine for me under Zesty.

 

 

*: Instead of rebooting, sudo modprobe -r iwlwifi && sudo modprobe iwlwifi will re-initialize the driver as well.

Use Apache 2.4.x as a secure reverse proxy for JBoss Wildfly

Published / by maurice / Leave a Comment

Quite some time ago, I wrote a post about using Apache as a secure reverse proxy for JBoss AS5. The development of both web servers and application servers has not come to a stand-still, so I felt it was time for a follow-up – not in the least because I have to configure such a set-up again myself.

This time around, Apache httpd has advanced to version 2.4 (the Ubuntu distro I’m using, 14.04 LTS, comes with version 2.4.7; the latest version available is 2.4.9); the JBoss application server has been going through versions AS6, AS7 and finally – reflecting a new naming scheme – Wildfly8 (I’m using version 8.1.0.Final).

For me, Apache is installed i/etc/apache2/, and you may install Wildfly anywhere (I’ll use {$wildfly-home-dir} to denote the path).

Note: The JBoss documentation expresses a preference for mod_cluster when putting the app server behind an Apache. However, this component is – as far as I can tell – only available for httpd 2.2.x (x >= 8), and trying to included the precompiled modules in an httpd 2.4 led to errors.

I decided to keep the default Apache installation and use mod_proxy_ajp instead; that was possible since I didn’t need the advantages mod_cluster advertises to have over its alternatives.

Securing the connection

To enable SSL security on the connection, enable the following parts in the /etc/apache2 directory by creating a symbolic link in the *-enabled subdirectories that point to the *-available subdirectories:

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/ssl.conf

 

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/ssl.load

 

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/socache_shmcb.load
  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/rewrite.load

 

  • /etc/apache2/sites-enabled$ sudo ln -s ../sites-available/default-ssl.conf

To make sure that all calls are made secure, redirect calls to the normal HTTP endpoints by adding an appropriate entry in the default virtual host configuration:

  • In /etc/apache2/sites-enabled/000-default.conf:
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

Note 1: The preferred RedirectPermanent directive didn’t seem to work properly for me. Maybe I just mis-configured something for that.

Note 2: An official SSL certificate needs to be purchased and installed to remove the warning browsers issue when landing on a page on the server. I’m not delving deeper into this issue here, see the mod_ssl description for details.

Enable the AJP protocol in Wildfly

To be accessible for the calls through the reverse proxy, Wildfly must expose a port on which it listens for traffic following the AJP protocol.

  • Add an entry to the {$wildfly-home-dir}/standalone/configuration/standalone.xml file, in the undertow subsystem within the default-server section:
    <name="ajpListener" scheme="http" socket-binding="ajp"/>

That’s all, because the corresponding socket binding is enabled by default (see bottom of that file), on port 8009.

Setting up Apache as a secure reverse proxy for Wildfly

Enable the following modules in order to be able to use mod_proxy_ajp:

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/proxy.conf

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/proxy.load

  • /etc/apache2/mods-enabled$ sudo ln -s ../mods-available/proxy_ajp.load

Activate the secure reverse proxy to the application server:

  • Add a proxying entry to the /etc/apache2/mods-enabled/proxy.conf file:
    ProxyPass / ajp://localhost:8009/
  • Enable secure proxying in the /etc/apache2/sites-enabled/default-ssl.conf file:
    SSLProxyEngine on

Taking it for a test run

Now fire up the application server:

  • {$wildfly-home-dir}/bin/standalone.sh

and (re)start the webserver:

  • sudo service apache2 restart

and go to the root of your installation (which may be  http://localhost/  if you’re trying this out locally). What you should see now is a warning from your browser, telling you the certificate that’s used by the site you’re trying to access is not trusted. If you choose to ignore this warning – you can, i.e. if you trust your own server – then you should be redirected to the landing page of the Wildfly installation (or anything you have deployed in the root context instead), served over a secure SSL connection.

‘External’ Quartz on JBoss 5.1

Published / by maurice / Leave a Comment

On our current project, we have the requirement to be able to run our application on more than JBoss AS alone. When confronted with the need to schedule certain tasks, we considered a few options:

  • Use EJB timers: Due to the nature of the tasks (e.g. interdependencies) this mechanism is not suitable – it’s simply just not sophisticated enough.
  • Use the Quartz functionality provided along with the JBoss distribution: While this may do exactly what we need, it wouldn’t be portable to app servers from other vendors.

So the decision was made to use Quartz, not as readily available from JBoss but as an add-on library (‘external’, if you will). I found a couple of articles that pointed me in the right direction:

We used Quartz version 2.0.2 – the latest and greatest at the time I write this – and since Maven is our build tool of choice, the following dependency pulls all required libs into our project:

<dependency>
    <groupid>org.quartz-scheduler</groupid>
    <artifactid>quartz-jboss</artifactid>
    <version>2.0.2</version>
    <scope>provided</scope>
</dependency>

We use provided scope here since we won’t be including Quartz in our project’s deliverables; instead we put such dependencies on our server explicitly. Either approach would work, though.

1) Clean up your installation.

So, the first job at hand is to remove the Quartz artifacts from our JBoss installation. I guess it makes sense to prevent different versions from showing up in your classpath. The files to be removed are:

  • ${jboss.home.dir}/common/lib/quartz.jar
  • ${jboss.home.dir}/server/[SERVER_NAME]/deploy/quartz-ra.rar (in EAP this is an exploded RAR, so remove the directory)

Before deleting anything from the common/lib directory, be sure that there aren’t any other servers running from the same AS installation that need that file!

2) Add the required Quartz libraries to you server.

The following file should be in place after this step:

  • ${jboss.home.dir}/server/[SERVER_NAME]/lib/quartz-all-2.0.2.jar

That’s right. Just one jar, very convenient – and the added bonus is that this already contains the stuff that’s needed for deployment in other app servers, so no need to have different dependencies in different deployments.

Note that this file is not downloaded if you include the dependencies through Maven as indicated above, you have to download the full distribution to get it.

3) Add an MBean to the JBoss configuration.

The following is a simple example of a Quartz MBean configuration:

<server>
    <mbean code="org.quartz.ee.jmx.jboss.QuartzService" name="user:service=QuartzService,name=QuartzService">
        <attribute name="JndiName">Quartz</attribute>
        <attribute name="Properties">
            org.quartz.scheduler.instanceName = DefaultQuartzScheduler
            org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
            org.quartz.threadPool.threadCount = 5
            org.quartz.threadPool.threadPriority = 4
            org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore
        </attribute>
    </mbean>
</server>

If you use this, no extra DataSource needs to be configured and Quartz keeps its jobs in memory. For more specifics on the configuration possibilities (there are a lot!), see http://www.quartz-scheduler.org/docs/configuration/.

Put the XML above in a x-service.xml file in your server’s deploy dir, like e.g.:

  • ${jboss.home.dir}/server/[SERVER_NAME]/deploy/quartz-service.xml

4) Point Quartz at the jobs you want done.

Obviously it now is possible to use Quartz from code inside deployed applications. Just retrieve the scheduler from JNDI like so:

InitialContext ctx = new InitialContext();
Scheduler scheduler = (Scheduler) ctx.lookup("Quartz");

But for our purpose this just isn’t good enough, we want to be able to schedule tasks from configuration files. To accomplish that, we need to perform a number of steps:

a) Enable the Quartz plugin that reads jobs and triggers from an indicated XML file. This is done by adding the following properties to the configuration shown in step 3):

org.quartz.plugin.jobInitializer.class = org.quartz.plugins.xml.XMLSchedulingDataProcessorPlugin
org.quartz.plugin.jobInitializer.fileNames = ${jboss.server.home.dir}/conf/quartz-jobs.xml
org.quartz.plugin.jobInitializer.scanInterval = 120

For details on this plugin see http://www.quartz-scheduler.org/api/2.0.0/org/quartz/plugins/xml/XMLSchedulingDataProcessorPlugin.html.

b) Create a class that implements the org.quartz.Job interface for each such a task. This interface exposes exactly one method, void execute(JobExecutionContext ctx), which is called when the job is triggered.

c) Provide the quartz-jobs.xml file that is indicated in the extra configuration in step a) (the file name and path can be adjusted to your liking) with the appropriate timing to start your jobs. For our tasks we use Cron-like jobs, with the Quartz CronTrigger (see http://www.quartz-scheduler.org/api/2.0.0/org/quartz/CronTrigger.html). An example configuration is:

<job-scheduling-data version="1.8" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.quartz-scheduler.org/xml/JobSchedulingData" xsi:schemalocation="http://www.quartz-scheduler.org/xml/JobSchedulingData http://www.quartz-scheduler.org/xml/job_scheduling_data_1_8.xsd">
 
    <pre-processing-commands>
        <delete-jobs-in-group>*</delete-jobs-in-group> 
        <delete-triggers-in-group>*</delete-triggers-in-group> 
    </pre-processing-commands>
    <processing-directives>
        <overwrite-existing-data>true</overwrite-existing-data>
        <ignore-duplicates>false</ignore-duplicates> 
    </processing-directives>
 
    <schedule>
        <job>
            <name>TestJob</name>
            <job-class>com.acme.quartz.TestJob</job-class>
        </job>

        <trigger>
            <cron>
                <name>TestCronTrigger</name>
                <job-name>TestJob</job-name>
                <cron-expression>0 0 3 ? * MON-FRI</cron-expression>
            </cron>
        </trigger>
    </schedule> 

</job-scheduling-data>

In this example the task executed by the com.acme.quartz.TestJob class is triggered at 3:00 AM on weekdays.

One last CAVEAT: On Windows, the ${jboss.server.home.dir} expression resolves to a String that contains backward slashes (‘\’) instead of forward slashes as path delimiter. The way the Quartz extension for JBoss reads in the properties is not able to cope with that, so you may need to provide a full path explicitly for any file names.

Use Apache as a secure (reverse) proxy for JBoss 5 AS/EAP

Published / by maurice / Leave a Comment

This task can be divided into two independent components (configure Apache to use SSL, set up Apache as a reverse proxy for JBoss) and a single step to make those two work together. The guidelines below have been successfully tested on an Apache 2.2.17/JBoss EAP 5.1.0.GA combination, the latter using Tomcat native libs, on a single server.

Part 1: Use SSL for access to Apache

1) Download and install the Apache Httpd server (version 2.2.6 or higher, 2.2.17 is the current). The folder in which the server is installed is referred to as APACHE_HOME further on.

2) In APACHE_HOME/conf/httpd.conf, un-comment the following lines:

    LoadModule ssl_module modules/mod_ssl.so

    Include conf/extra/httpd-ssl.conf

Then comment the following one (to restrict access without SSL):

    #Listen 80

3) Put your certificate and key in the APACHE_HOME/conf folder, and (if necessary) change the names in APACHE_HOME/conf/extra/httpd-ssl.conf entries to match:

    SSLCertificateFile "[APACHE_HOME]/conf/server.crt"
    SSLCertificateKeyFile "[APACHE_HOME]/conf/server.key"

If you don’t have a CA certificate, you can create a self-signed certificate for testing purposes, see e.g. the OpenSSL FAQ how to do so. An OpenSSL executable is provided in the APACHE_HOME/bin folder.

Remark: On Windows platforms it is not possible to use the SSLPassPhraseDialog-parameter (in httpd-ssl.conf) with the default value ‘builtin’. The simplest (albeit not the safest) solution is to remove the passphrase from the key, removing the need for Apache to ask for it at startup.

4) (Re-)start the Apache server and test whether it works as expected… and don’t forget the ‘https://’!

For an extensive explanation of the SSL configuration possibilities see Apache Module ssl_mod.

Part 2: Set Apache up as a reverse proxy for JBoss

1) Download the mod_jk connector (version 1.2.15 or higher, 1.2.31 is the current), rename the ‘mod_jk-1.2.[*]-httpd-2.2.x.so’ file to ‘mod_jk.so’ and move it to the APACHE_HOME/modules folder.

2) Add the following line to APACHE_HOME/conf/httpd.conf:

    Include conf/mod-jk.conf

3) Create a new file in APACHE_HOME/conf with the name ‘mod-jk.conf’, and fill it with:

    LoadModule jk_module modules/mod_jk.so

    JkWorkersFile conf/workers.properties

    JkLogFile logs/mod_jk.log
    JkLogLevel info
    JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"
    JkRequestLogFormat "%w %V %T"

    JkOptions +ForwardKeySize +ForwardURICompatUnparsed -ForwardDirectories

    # Mount your applications
    ###JkMount /application/* loadbalancer
    # Mount all URLs:
    JkMount /* node1

    # You can addionally use external file for mount points.
    ###JkMountFile conf/uriworkermap.properties
    # Mount file reload check interval in secs (0 = turned off).
    ###JkMountFileReload 60

    # Add shared memory. Used only on unix platforms. The shm file is used by balancer and status workers.
    ###JkShmFile run/jk.shm

    # Add jkstatus for managing runtime data:
    <Location /jkstatus/>
        JkMount status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
    </Location>

When not all requests are to be redirected to node1 the line starting with ‘JkMount’ must be adjusted. Furthermore it is possible to use a separate properties file (using ‘JkMountFile’, with entries following the pattern ‘URL=worker’, e.g. ‘/jmx-console=node1’) if you need a more extensive redirection scheme.

In the configuration above the access to the status manager (worker with ID ‘status’) is restricted to clients running on the same host, just for illustrative purposes.

See the Tomcat connector reference for further details and possibilities.

4) Create a new file in APACHE_HOME/conf named  ‘workers.properties’,  and put the following in it:

    # Define list of workers that will be used for mapping requests

    # Define Node1
    # modify the host as your host IP or DNS name.
    worker.node1.type=ajp13
    worker.node1.host=localhost
    worker.node1.port=8009
    worker.node1.ping_mode=A
    #worker.node1.connection_pool_size=10 # Only if the number of allowed connections to the Httpd is higher than maxThreads in JBoss server.xml.
    #worker.node1.lbfactor=1 # Only used for a member worker of a load balancer.
    # For non-loadbalanced setup with a single node:
    worker.list=node1

    # Define Node2
    # modify the host as your host IP or DNS name.
    #worker.node2.type=ajp13
    #worker.node2.host= node2.mydomain.com
    #worker.node2.port=8009
    #worker.node2.ping_mode=A
    #worker.node2.connection_pool_size=10
    #worker.node2.lbfactor=1

    # Load-balancing behaviour
    #worker.loadbalancer.type=lb
    #worker.loadbalancer.balance_workers=node1,node2
    #worker.loadbalancer.sticky_session=Off # Enabled by default.
    #worker.list=loadbalancer

    # Status worker for managing load balancer
    worker.status.type=status
    worker.list=status

Most lines above are commented out, since we’re aiming for a configuration for a single node without loadbalancing. It is straightforward to add more nodes, with or without loadbalancing; just pay attention to the fact that with loadbalancing the worker.list should not refer to the separate nodes but only to the loadbalancer worker.

5) For each (JBoss-)node a ‘jvmRoute’ attribute  must be added to the <Engine>-element in JBOSS_HOME/server/[configuration]/deploy/jbossweb.sar/server.xml, using the corresponding name from the mod_jk-configuration as a parameter:

    <Engine name="jboss.web" defaulthost="localhost" jvmroute="node1">

And for JBoss AS/EAP version 5 and above that is all that is required!

6) If you didn’t configure Apache to use SSL, you can now (re-)start the JBoss en Apache servers and test whether the redirecting functions as expected…

If you did configure SSL for Apache, hang on just a bit more…

Part 3: Combining the two solutions above

To be able to access Apache using SSL after which the request is passed to the JBoss instance over AJP, one last adjustment is required:

1) Move the JkMount directives from the APACHE_HOME/conf/mod-jk.conf file to the APACHE_HOME/conf/extra/httpd-ssl.conf file, and make sure they’re within the <VirtualHost> tags:

<VirtualHost _default_:443>

    […]

    JkMount /* node1
    <Location /jkstatus/>
        JkMount status
        Order deny,allow
        Deny from all
        Allow from 127.0.0.1
    </Location>

</VirtualHost>

After a restart of the Apache server the pages served by JBoss will be available over HTTPS from Apache (port 443).

Be aware that they are also still available over HTTP from JBoss directly (on port 8080), since the configuration above didn’t remove that (default) situation. To accomplish that, you should comment the HTTP connector entry in the server.xml file of jbossweb.sar.

“Too many certificates in chain”? It just may be a corrupt keystore!

Published / by maurice / Leave a Comment

Update: This problem relates to JDK 6 and earlier, and is handled differently in JDK 7 and up.

Recently we ran into some trouble with the keyword expansion functionality that CVS offers (way too much false positives in the comparisons), and we simply decided to turn it off for our sources. That meant changing the ASCII/Binary property of all files from “ASCII -kkv” to “ASCII -kk” (keyword compression). Problem solved, albeit in a rather crude way.

Well, that sure bit us in the proverbial back end. The point is that when you change this property, you should be careful not to change it for files that were designated “binary”. But we did.

One side effect of this was that a keystore file, stored in CVS, now became ASCII as well; effectively corrupting the file for further use. When trying to read a key from it, I got the following stack trace:

Caused by: java.io.IOException: Too many certificates in chain
   at sun.security.provider.JavaKeyStore.engineLoad(Unknown Source)
   at java.security.KeyStore.load(Unknown Source)

In an attempt to locate the source for this, I stumbled upon this:

sun/security/provider/JavaKeyStore.java#engineLoad(InputStream, char[])

(just scroll down a bit, the top bar covers the important part!)

The first exception that is documented tells us:

java.io.IOException: if there is an I/O or format problem with the keystore data

So from the code in that method, I gather that the occurring problem is actually an OutOfMemoryError (which I feel is kind of creepy) that is caused by the keystore implementation trying to load a corrupted keystore file. I think it’s unlikely that there will be so many chained certificates in any practical keystore that this will really lead to running out of memory, so next time I see this error message I surely will think ‘keystore file corruption’!

An inconsistency between explicit and implicit hashing when signing in Java security?

Published / by maurice / Leave a Comment

For the connection to a certain other system within our network, the program I’m working on needs to verify that it indeed is what it claims to be: an authorized client. A common way to accomplish is through PKI: it signs the message it sends using a private key, and the other system can verify this signature using the corresponding public key. See e.g. this article for an explanation of how this works.

In our case, there are three steps in signing a message:

  • calculation of the message digest through a hashing algorithm,
  • calculation of the digital signature using the private key, and
  • coding the result to base64.

The last step is not part of the normal signing process, but we need to send the result as a string inside an XML message. Using the ‘raw’ signature would result in weird characters in the XML, very likely choking up the parser.

As I was coding away, I was lulled into performing each of these steps separately, so I started off with implementing the hashing using the java.security.MessageDigest class. I instantiated it with the “SHA-1” algorithm and simply called the digest method with the message to obtain

its hash. Pretty straightforward stuff.

Then I turned to the java.security.Signature class to supply me with the subsequent signing functionality. It occurred to me that there are algorithm choices that include hashing algorithm names, so I quickly found out that it is possible to let the Signature class take care of

both the first and second step of my signing process. While that struck me as quite convenient, I decided to stick to the original plan and not use the hashing possibility here. I chose the “NONEwithRSA” algorithm, and after feeding the message and private key the sign method provided me with an answer.

Then I encoded it in base64 (using the Apache Commons Codec library, which I also could have used for the hashing functionality) and presto! So I thought, at least…

But then…

The first test we performed immediately indicated something was wrong. And after checking everything else (like making sure code page encodings were correct and what not) we came to the conclusion that the signature itself had to be the culprit.

So I decide to put the ‘my’ way of creating a signature side by side with the signing method using the implicit hashing possibility, to see whether there might be a difference in the outcome:

public void test(byte[] data, PrivateKey privateKey) throws Exception {
   // Explicit hash and separate signing:
   byte[] hashedData = MessageDigest.getInstance("SHA-1").digest(data);
   byte[] signedData = signData(hashedData, privateKey, "NONEwithRSA");

   // Signing with implicit hashing:
   byte[] signedHashedData = signData(data, privateKey, "SHA1withRSA");

   System.out.println("Encoded data (explicit hashing) = "
           + new String(Base64.encodeBase64(signedData)));
   System.out.println("Encoded data (implicit hashing) = "
           + new String(Base64.encodeBase64(signedHashedData)));
}

private static byte[] signData(byte[] data, PrivateKey privateKey, String algorithm) throws Exception {
   Signature signature = Signature.getInstance(algorithm);
   signature.initSign(privateKey);
   signature.update(data);
   return signature.sign();
}

And even though:

  • from the code it seems that both paths should lead to the same result: no other configuration than the algorithm names are given, so all else should be default, and
  • from the Java security documentation for the signing options “NONEwithRSA” is stated to ‘not use a digesting algorithm’, so it should act as “SHA1withRSA” without the SHA-1 hashing,

there definitely is a difference between the outcomes!

In our situation, the implicit hashing turned out to deliver the correct result (at least, with regards to what the other system expected), so a minimal code change (getting rid of the explicit hashing step) did the trick. We use an external configuration file to set the signing algorithm through a system property, so changing that was easy.

What’s causing this?

Now why is there a difference between the two approaches? I’ve tried to find an answer using Google, but that quest didn’t turn up any answers.

So I did what any self-respecting developer would do: step through the implementation in a debugger. Unfortunately my toolkit didn’t allow me to see everything I wanted to; however I could see that the input of the signing step was identical in both cases and the same implementation for signing is used under the hood (sun.security.rsa.RSACore). I cannot see what is happening with respect to internediate padding of the byte arrays, however, so I’m guessing that the ‘default’ settings of the two approaches – driven by different SignatureSpi implementations – differ in this respect.

If anyone could point out the actual difference to me, that would be greatly appreciated. For now I’ll have to be content with knowing that the two approaches do return different results and that picking one at random may lead to problems…