I was learning about OFX api's (what Quicken uses to pull your transaction info) for a personal project, and I kept getting 400 Bad Request errors.  I wanted to see what the requests I was making with curl looked like to the server.  Then I wanted to compare what requests from an existing tool looked like.  This led me to socat.

This won't really be a MITM in this post, since we won't be forwarding the traffic on.  I'm sure this is something socat could do, but for my use, I was just interested in being a "sink" of traffic.

What is socat?

From socat's manpage:

Socat is a command line based utility that establishes two bidirectional byte streams and transfers data between them. Because the streams can be constructed from a large set of different types of data sinks and sources (see address types), and because lots of address options may be applied to the streams, socat can be used for many different purposes.

A more concise definition is from copyconstruct:

socat stands for SOcket CAT. It is a utility for data transfer between two addresses.

So it's useful you've got some data on one address, and you want to move it to another address.  In my case, I wanted to take data from a curl request, and spit it to STDOUT.

Simple example: curl --> stdout

In one terminal:

socat -u TCP-LISTEN:8000 STDOUT

In another terminal

curl localhost:8000 --data @somefile.txt

And in the socat terminal, we'll see:

POST /hi/there HTTP/1.1
Host: localhost:8000
User-Agent: curl/7.68.0
Accept: */*
Content-Length: 1679
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue

<contents of the file>

The arguments to socat are:

  • -u – make the first address read-only.
  • TCP-LISTEN:8000 – listen on port 8000 for TCP connections
  • STDOUT – take everything from the first address and send it to standard out.

The interesting thing for me was how newlines were treated.  In my case, it was all one line, which I suspected was part of my problem, since the request was an old OFX format (pre-XML) that didn't have closing tags.

I wanted to be sure, so I found a library called ofxgo that has a command line utility for making OFX requests.  However, I had one issue: it refused to send the OFX request over a plaintext connection.  It required an HTTPS address, a totally valid requirement when working with sensitive financial credentials.

socat with HTTPS

Fortunately, socat can make TLS connections!  Starting from this example, I ended up with the following:

$ cert() {
   openssl genrsa -out $1.key 2048
   # Note: answer localhost for your Common Name (CN)
   # other answers don't really matter
   openssl req -new -key $1.key -x509 -days 3653 -out $1.crt
   cat $1.key $1.crt > $1.pem
}
$ cert server

$ openssl dhparam -out dhparams.pem 2048
$ cat dhparams.pem >> server.pem

The example is set up for mutual TLS (i.e. the client validates the server's identity and the server validates the client identity), so I removed the client cert generation.

Next I started socat like this:

socat ssl-l:1443,reuseaddr,fork,cert=server.pem,verify=1  STDOUT

In this example, we're listening on port 1443, and we use the server's .pem file as our identity.  Notice I set verify=1. This turns off client cert verification, so only the client is validating the server.

Then in curl:

curl -v  --cacert server.crt https://localhost:1443

Setting CA's more broadly

My next issue: ofxgo didn't have a way to specify a Certificate Authority through a command line flag or environment variable. (I tried a few permutations of SSL_CERT_FILE and SSL_CERT_DIR to no avail).

Luckily there was a StackOverflow post with instructions on how to add a root CA on linux:

sudo mkdir /usr/share/ca-certificates/extra
sudo cp server.crt /usr/share/ca-certificates/extra/server.crt
sudo dpkg-reconfigure ca-certificates

With that in place, I could curl without specifying --cacert, and I could successfully point ofxgo at https://localhost:1443.