Red Green Refactor

Again, some things from Apple could be done easier…here is the story.

Problem: Your iPhone app needs to talk to a web app which responds only over SSL (so you have to use https url scheme). In testing, you do not want to use the real service, because sometimes it costs you money, and, well, it just does not seem right. Of course you may create a server that responds on http and then use that instead of using secure connection, but it would mean you will have change your app just before releasing it. The same would happen if you follow some of the answers posted to this StackOverflow question where many advise disabling certificate verification programmatically when your app is still in development. All right, maybe you do not release your app after every single build, but still you give it to testers and so on. And while forgetting changing http back to https would just make your app rejected, forgetting to turn on certificated verification is opening your app to man-in-the-middle (my favorite) attack. I was searching desperately to some good quick but elegant solution to this problem. Fortunately, on the StackOverflow topic mentioned above I found some very helpful clue.

Your own CA ?

As Ron Gutierrez says on GDS security blog, when you install certificate on your iOS device (just send your cert file as an email attachment, open email, click on your certificate, and then confirm by hitting install) it adds a record to the tsettings table in the TrustStore.sqlite3 database. As Ron discovers further, the same table, the same database in the same file is used by your iPhone Simulator. So, you have to add a similar record to the tsettings table inTrustStore.sqlite3 database which for iPhone Simulator 5.1 can be found here on your Mac:

~/Library/Application Support/iPhone Simulator/
    5.1/Library/Keychains/

Ron even provides a script in Python which given your CA cert file will do the dirty job to add what is necessary to the tsettings table. Well, at least it did some time ago, because the blog entry is more than one year old from iPhone Simulator version 4.3. Now when I look at the contents of the Library folder iPhone Simulator 4.3 the folder Keychains is not even present (maybe this is because I did not run iPhone Simulator 4 since long time). The script seems to alter the tables for more recent simulators, but unfortunately, when doing a simple synchronous request using NSURLConnection it fails miserably with the message:

Error: [('SSL routines', 'SSL23_READ',
    'ssl handshake failure')]

Well, something is wrong: either my certificate does not work, or Ron’s script is outdated. First things, first. Because I am using my own CA (rather than PortSwigger CA mentioned in the Ron’s post) it may well be the case that my certificates are not good.

Setting up a local HTTPS server

Using webapp2 and Paste [httpserver][5] is quick and easy to run a server.

httpserver.serve(app, host='yourhostname',
    port='443', ssl_pem='server.pem')

where app is your WSGI Application and server.pem is your cert file (signed by your CA) and server key concatenated in one file.

Creating CA, keys and certificates

And so we need a CA. I found a very nice description of how to do that [here][6]. For convenience, here is a short summary:

1. Create a key for your CA: openssl genrsa -des3 -out ca.key 4096

2. Create a CA certificate:

   openssl req -new -x509 -days 365 -key ca.key
       -out ca.crt
   

   Here, you will need to answer a couple of questions. One field seems to be important here: YOURNAME (more commonly known as Common Name – CN for short). It should be the fully qualified domain name of the web service you want to mock out – the same what you use in your requests with NSURLConnection.

   Actually I did it a bit differently on my setup. I use the same Common Name in both CA and server certificates, but I add CA to the Organisation field so the two descriptions are still different. Maybe someone can check if it really matters, or I will when it bites me again.

3. Now, more or less the same for the server keys:

   openssl genrsa -des3 -out server.key 4096
   openssl req -new -key server.key -out server.csr
   

4. Sign the server certificate signing request (server.csr in the command above) with the self-created Certificate Authority (CA) that you made earlier.

   openssl x509 -req -days 365 -in server.csr -CA ca.crt
       -CAkey ca.key -set_serial 01 -out server.crt
   

5. Create insecure version (without password protection) of your server key, so that you do not have to type password every time you start your server.

   openssl rsa -in server.key -out server.key.insecure
   mv server.key server.key.secure
   mv server.key.insecure server.key 
   

   NOTE: you should be able to use rsa keys for CA key as well, but I did not check it.

6. Create a pem file that we use in Paste.httpserver above.

   cat server.crt server.key > server.pem
   

Now run the server and try to connect to it with your browser. Before, you need to do two things:

1. You will need to modify your /etc/hosts file to point the simulated domain to localhost.
2. You have to add your CA certificate to Keychain access by double clicking your ca.crt file in Finder if you use Safari or by going to Preferences => Advanced => Encryption => View certificates => Import if you use Firefox. Do not forget to confirm that your trust your certificate.

Now, you should be able to connect to your server using https and your browser should not complain about anything. Finally, you may want to check if your CA certificate works your iOS device. To do so, however, you need to create another set of certificates that will use your IP address as your Common Name. If your MAC is on 192.168.0.56 then you will have to use 192.168.0.56 CA as Common Name for your CA certificate and 192.168.0.56 for your server. This calls for a helper script: who wants to write it ? Once you mail and add your CA cert on your iPhone, you should be able to connect to https://192.168.0.56 from your iPhone and it should not complain.

Back to iPhone Simulator

If we are here, it means that our CA is setup correctly, which means we have to dig into TrustStore.sqlite3 file. If it works on iPhone and if its true that iPhone adds the information about your CA certificate to TrustStore.sqlite3, there is still hope. We should be able to look into TrustStore.sqlite3 on the iPhone, find the added record, and add exactly the same record to the iPhone Simulator equivalent. How to extract TrustStore.sqlite3 from your iPhone ? Fortunately, people did it before: this StackOverflow question provides a link to a tool called iPhone Backup Extractor. What you have to do is:

1. Backup your iPhone after adding your CA certificate.
2. Run the iPhone Backup Extractor app, select your backup from the list and then extract the iOS Files group.

3. After extracting your will find the TrustStore.sqlite3 in the folder you specified after hitting Extract button. Having the TrustStore.sqlite3 file, we just have to verify that there is an entry corresponding to our certificate. I am using Firefox sqlite plugin to access the database. So I opened my iPhone Simulator TrustStore.sqlite3 file and the file on the iPhone. In the group Tables you will find the tsettings table and there you will indeed find some records (number of entires will probably correspond to the number of configuration profiles on your iPhone). On his blog, Ron says that only the first column: sha1 matters (and you should see that Ron’s script is handling the sha1 column properly – it is the same on the iPhone and on the iPhone Simulator for the same certificate – good), and in his script he fills the remaining columns with random blobs. It seems this assumption does not hold for iOS 5 as there I see that other columns are not even of the same length. If you add more certificates your will see that actually the entires are different for each certificate.

   I first realized that this may be the problem when I look at this blog post. I have downloaded their tool where they hard coded the whole tsetting record for their certificate and it looked different from what I see in the Ron’s script.

   Now, you have to copy the record that corresponds to your certificate from the database on iPhone to the database on iPhone Simulator. I used Firefox SQLite plugin again. And know big test. Run your Xcode project on you iPhone simulator (maybe you need to quit simulator if it is running to make sure it loads new data) and it should work now. One more question I have is: isn’t it all crazy ?

Tags: https, iPhone Simulator, ssl, testing

This entry was posted on Thursday, June 7th, 2012 at 12:26 pm and is filed under Engineering. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.