Testing HTTPS on iPhone Simulator

Again, some things from Apple could be done easier…here is the story.

Problem: Your iPhone app needs to talk to a web app which responds only over SSL (so you have to use https url scheme). In testing, you do not want to use the real service, because sometimes it costs you money, and, well, it just does not seem right. Of course you may create a server that responds on http and then use that instead of using secure connection, but it would mean you will have change your app just before releasing it. The same would happen if you follow some of the answers posted to this StackOverflow question where many advise disabling certificate verification programmatically when your app is still in development. All right, maybe you do not release your app after every single build, but still you give it to testers and so on. And while forgetting changing http back to https would just make your app rejected, forgetting to turn on certificated verification is opening your app to man-in-the-middle (my favorite) attack. I was searching desperately to some good quick but elegant solution to this problem. Fortunately, on the StackOverflow topic mentioned above I found some very helpful clue.

Your own CA ?

As Ron Gutierrez says on GDS security blog, when you install certificate on your iOS device (just send your cert file as an email attachment, open email, click on your certificate, and then confirm by hitting install) it adds a record to the tsettings table in the TrustStore.sqlite3 database. As Ron discovers further, the same table, the same database in the same file is used by your iPhone Simulator. So, you have to add a similar record to the tsettings table inTrustStore.sqlite3 database which for iPhone Simulator 5.1 can be found here on your Mac:

~/Library/Application Support/iPhone Simulator/
    5.1/Library/Keychains/

Ron even provides a script in Python which given your CA cert file will do the dirty job to add what is necessary to the tsettings table. Well, at least it did some time ago, because the blog entry is more than one year old from iPhone Simulator version 4.3. Now when I look at the contents of the Library folder iPhone Simulator 4.3 the folder Keychains is not even present (maybe this is because I did not run iPhone Simulator 4 since long time). The script seems to alter the tables for more recent simulators, but unfortunately, when doing a simple synchronous request using NSURLConnection it fails miserably with the message:

Error: [('SSL routines', 'SSL23_READ',
    'ssl handshake failure')]

Well, something is wrong: either my certificate does not work, or Ron’s script is outdated. First things, first. Because I am using my own CA (rather than PortSwigger CA mentioned in the Ron’s post) it may well be the case that my certificates are not good.

Setting up a local HTTPS server

Using webapp2 and Paste [httpserver][5] is quick and easy to run a server.

httpserver.serve(app, host='yourhostname',
    port='443', ssl_pem='server.pem')

where app is your WSGI Application and server.pem is your cert file (signed by your CA) and server key concatenated in one file.

Creating CA, keys and certificates

And so we need a CA. I found a very nice description of how to do that [here][6]. For convenience, here is a short summary:

  1. Create a key for your CA: openssl genrsa -des3 -out ca.key 4096
  2. Create a CA certificate:

    openssl req -new -x509 -days 365 -key ca.key
        -out ca.crt
    

    Here, you will need to answer a couple of questions. One field seems to be important here: YOURNAME (more commonly known as Common Name – CN for short). It should be the fully qualified domain name of the web service you want to mock out – the same what you use in your requests with NSURLConnection.

    Actually I did it a bit differently on my setup. I use the same Common Name in both CA and server certificates, but I add CA to the Organisation field so the two descriptions are still different. Maybe someone can check if it really matters, or I will when it bites me again.

  3. Now, more or less the same for the server keys:

    openssl genrsa -des3 -out server.key 4096
    openssl req -new -key server.key -out server.csr
    
  4. Sign the server certificate signing request (server.csr in the command above) with the self-created Certificate Authority (CA) that you made earlier.

    openssl x509 -req -days 365 -in server.csr -CA ca.crt
        -CAkey ca.key -set_serial 01 -out server.crt
    
  5. Create insecure version (without password protection) of your server key, so that you do not have to type password every time you start your server.

    openssl rsa -in server.key -out server.key.insecure
    mv server.key server.key.secure
    mv server.key.insecure server.key 
    

    NOTE: you should be able to use rsa keys for CA key as well, but I did not check it.

  6. Create a pem file that we use in Paste.httpserver above.

    cat server.crt server.key > server.pem
    

Now run the server and try to connect to it with your browser. Before, you need to do two things:

  1. You will need to modify your /etc/hosts file to point the simulated domain to localhost.
  2. You have to add your CA certificate to Keychain access by double clicking your ca.crt file in Finder if you use Safari or by going to Preferences => Advanced => Encryption => View certificates => Import if you use Firefox. Do not forget to confirm that your trust your certificate.

Now, you should be able to connect to your server using https and your browser should not complain about anything. Finally, you may want to check if your CA certificate works your iOS device. To do so, however, you need to create another set of certificates that will use your IP address as your Common Name. If your MAC is on 192.168.0.56 then you will have to use 192.168.0.56 CA as Common Name for your CA certificate and 192.168.0.56 for your server. This calls for a helper script: who wants to write it ? Once you mail and add your CA cert on your iPhone, you should be able to connect to https://192.168.0.56 from your iPhone and it should not complain.

Back to iPhone Simulator

If we are here, it means that our CA is setup correctly, which means we have to dig into TrustStore.sqlite3 file. If it works on iPhone and if its true that iPhone adds the information about your CA certificate to TrustStore.sqlite3, there is still hope. We should be able to look into TrustStore.sqlite3 on the iPhone, find the added record, and add exactly the same record to the iPhone Simulator equivalent. How to extract TrustStore.sqlite3 from your iPhone ? Fortunately, people did it before: this StackOverflow question provides a link to a tool called iPhone Backup Extractor. What you have to do is:

  1. Backup your iPhone after adding your CA certificate.
  2. Run the iPhone Backup Extractor app, select your backup from the list and then extract the iOS Files group.
  3. After extracting your will find the TrustStore.sqlite3 in the folder you specified after hitting Extract button. Having the TrustStore.sqlite3 file, we just have to verify that there is an entry corresponding to our certificate. I am using Firefox sqlite plugin to access the database. So I opened my iPhone Simulator TrustStore.sqlite3 file and the file on the iPhone. In the group Tables you will find the tsettings table and there you will indeed find some records (number of entires will probably correspond to the number of configuration profiles on your iPhone). On his blog, Ron says that only the first column: sha1 matters (and you should see that Ron’s script is handling the sha1 column properly – it is the same on the iPhone and on the iPhone Simulator for the same certificate – good), and in his script he fills the remaining columns with random blobs. It seems this assumption does not hold for iOS 5 as there I see that other columns are not even of the same length. If you add more certificates your will see that actually the entires are different for each certificate.

    I first realized that this may be the problem when I look at this blog post. I have downloaded their tool where they hard coded the whole tsetting record for their certificate and it looked different from what I see in the Ron’s script.

    Now, you have to copy the record that corresponds to your certificate from the database on iPhone to the database on iPhone Simulator. I used Firefox SQLite plugin again. And know big test. Run your Xcode project on you iPhone simulator (maybe you need to quit simulator if it is running to make sure it loads new data) and it should work now. One more question I have is: isn’t it all crazy ?

Tags: , , ,

13 Responses to “Testing HTTPS on iPhone Simulator”

  1. Hiedi says:

    So I was able to get SSL working in the ios 5.0 and ios 4.3 simulator using your steps and the info on the charles prozy site but its not working in ios 5.1 or 6.0. Any ideas on how to get it working again?

    • Hiedi says:

      Scratch that. Appears that the API I am trying to SSL with is the problem. Works against the prod version, not against the QA version….Hmmmm.

    • marcin.czenko says:

      Hi,

      It was tested with ios 5.1 so far and I haven’t upgraded yet to the beta version of new Xcode with ios6. I do not want to do it now, but I will be upgrading soon. If you already know the solution by that time, please share it.

  2. Vikas says:

    Hi Guys,
    This is not working on iPhone Simulator 6.0

    Any ideas?

  3. Steven says:

    Is there any progress on the ios6 simulator version?

  4. daniel-c says:

    There is now a script that works for iOS 5.0 to 6.1, allowing to directly import CA certificate to the simulator. Check https://github.com/ADVTOOLS/ADVTrustStore

  5. Dave says:

    Is there already a solution for the 6.0/6.1 simulator?

  6. marcin.czenko says:

    I am sorry for not reporting anything useful yet. I did not have yet time to look at it. We are doing some other work at the moment, but soon will move back to our favourite area. I would appreciate if you keep the thread updated. The info from daniel-c looks very promising.

  7. Kenni says:

    Just FYI: The ADVTrustStore tool posted by daniel-c works extremely well with the 6.0/6.1 emulator, it even allows you to delete the old invalid certificates inserted by add_ca_to_iossim.py :)

  8. Logan says:

    I’m having trouble getting a certificate to install on the simulator. Well correction: it appears to install fine, but has no effect. I’m still getting “The certificate for this server is invalid…” on the simulator. I don’t know if I’m not giving it the right certificate/format, or if it’s just not being recognized. I generated the cert file using openssl as per http://artur.ejsmont.org/blog/content/how-to-import-self-signed-ssl-certificate-to-java-keystore-adding-https-certificate but I don’t even know if that’s the right kind of certificate to install. Any help would be appreciated.

    • marcin.czenko says:

      Hi Logan,

      Did you get any progress on this. I’ve just tested the whole procedure with new Xcode and iPhone Simulator v5.1 and v.6.1 – Many thanks to Kenni for mentioninig the ADVTrustStore. It works fine but I see that the post needs to be updated here and there. Maybe I will be able to help with certificates, but maybe in the meantime you fixed the issue yourself.

      Cheers,
      Marcin

Leave a Reply