Basic authentication with FileMaker Insert from URL

We received an email this morning asking for some assistance with cURL for FileMaker. Paraphrased they were asking

I am trying to get the results of a webpage using INSERT FROM URL into a text field, but the web developer has set the site to require a username / password via basic http authentication. They are also expecting the user name and password to be base 64 encoded.

If I manually load the URL in a browser, it asks for a user name and password and then when I enter these, it loads the page to give the data that I need. But, I can’t figure out how to do this inside FileMaker.

While we’ve provided all sorts of Insert from URL examples in sessions materials for FileMaker DevCon and other presentations (like here, here, and here) I couldn’t find anywhere in those example materials that used Basic Auth. Given how prevalent it is in the world of the web that seems like something of an oversight which I’m correcting here. The demo file has the script which you can copy and paste from as you need.

Update

As a couple of people have pointed out there’s actually an easier way – isn’t there always with FileMaker! cURL itself will actually take care of the base 64 encoding for us – so while the way I’ve described below sure does work, using the built-in mechanism makes things easier still.

Here’s the super quick way

Set Variable [ $username ; Value: "username" ]
Set Variable [ $password ; Value: "password" ]
Set Variable [ $curlOptions ; Value: "--user " & quote ( $username & “:” & $password ) ] 
Set Variable [ $url ; Value: "https://the.website.com/path" ]
Insert from URL [ Select ; With dialog: Off ; Target: $response ; $url ; cURL options: $curlOptions ]

The original

We can perform a Basic Auth login by sending an additional header in the request. This takes the form

--header "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ="

As the email we received pointed out, the username and password need to be base 64 encoded, and that’s what dXNlcm5hbWU6cGFzc3dvcmQ= is. And it’s slightly more than the username and password – you also need a colon ‘:’ between them. Assuming your username is username and your password is password then you need

base64encode('username:password')

Now that’s all well and good, but how do we do that in FileMaker? Fortunately FileMaker provides a script step to perform base 64 encoding – in fact it provides two of them! There’s Base64Encode and then there’s Base64EncodeRFC.

At first glance it looks like the first of these will solve the problem for us, and what’s this RFC bit about?

The original Base64Encode step first turned up in v13 and almost does what we need, but it comes with one slight drawback – it follows RFC 2045 (more on that in a second) which means that it has a limit of 76 characters per line, and then adds a CRLF (carriage return, linefeed) character.

You get a CRLF every 76 characters as well as at the very end of the hash (even if it has fewer than 76 characters). Unfortunately this will cause you no end of problems in a basic auth header, so we either need to strip out the CRLFs or do something differently.

The better solution is ‘do something different’ and use Base64EncodeRFC. The ‘RFC’ bit of that is the acronym Request For Comment – which isn’t entirely how it sounds either.

While an RFC starts out as a request for comment, once it had been ratified it becomes a standard. In this context the RFC was issued by the ITEF – the Internet Engineering Task Force – you can read more about them if you wish, but essentially they have drawn up specifications on exactly how the output of the base 64 encoding should be formatted.

The FileMaker Base64EncodeRFC supports five different RFCs which are listed in the step documentation in the FileMaker help, the one we need in this case is 4648 which has no limit on line length, and no pesky line breaks to muck things up.

So to generate the necessary hash in FileMaker we can use

Set Variable [ $username ; Value: "username" ]
Set Variable [ $password ; Value: "password" ]

Set Variable [ $auth ; Value: Base64EncodeRFC ( 4648 ; $username & ":" & $password ) ]

Now that we have the hash that we need, that needs to be set as a header which can be used in the Insert from URL cURL options. We need to be a little bit careful in how we do that because the header content does need some spaces in it, but we can’t have any spaces in the header string itself. As always with FileMaker there are a couple of ways to do this, using either Set Text or making sure we escape the quotes. Personally I prefer the later approach, which ends up looking like this

...continuing form above

Set Variable [ $curlOptions ; Value: "--header \"Authorization: Basic " & $auth & "\" " ]

See how the set of quotes just before the word Authorization has the \ before it – that prevents FileMaker from seeing it as the end of the string, and actually results in it being in the string when it gets parsed. Note also that I’ve left a space at the end of the line, just after the second escaped quote – that allows us to add further options in this variable if we wish – see the example script for a few other useful options to include in here.

Oh, and for those of us who aren’t from North America – remember to spell Authorization incorrectly with a ‘z’, not properly with an ‘s’ (hey I live in England, I know how English is supposed to be ;-)) otherwise it doesn’t work either!

With that in place we can go ahead and make the Insert from URL call

...continuing form above

Set Variable [ $url ; Value: "https://the.website.com/path" ]
Insert from URL [ Select ; With dialog: Off ; Target: $response ; $url ; cURL options: $curlOptions ]

All going well, the $response variable now contains the content of the URL which you called, having successfully sent the authorisation header and ‘logged you in’ in the process.

All of this is put together in one copy-and-paste script in the demo file. Hopefully this helps get people headed in the right direction – feel free to get in touch if you run into issues!