Our New Server, and Internet Services

Last Sunday, we migrated Gimlet to a big(!) shiny new server. All of the associated hiccups seem to have been solved. We always think the new server is so huge, and then outgrow it surprisingly fast.

In this migration, we moved from our own dedicated server at Rimuhosting to a cloud host at Digital Ocean. We were very happy with Rimuhosting’s service, but the flexibility of the cloud servers is a huge advantage for us. It’ll be a lot easier for us to grow here. The price is rather lower, too, which doesn’t hurt.

Doing the migration was a bunch of work, but since the service is a better fit for us, it was worth the time spent.

Of course, as a service provider ourselves, this is a somewhat sobering realization — but a valuable one to keep in our minds.

December 12th, 2014  |  Published in Gimlet


New Prices and Terms of Use

More than six months ago, we announced Gimlet’s new pricing: $20 per month for your first branch, and $10 per month for each additional. As of September 1, those prices have been in effect. If you’re billed on a monthly cycle, you’ll see the increase in this month’s receipt; if you’re billed annually, you’ll see it on your next invoice.

With this change, we needed to update our Terms of Use document, as it mentions prices. Among some legan and readability tweaks (no more all caps paragraphs!), there are a few other changes of note in there:

  • You’re now formally allowed to be billed annually and pay by check.
  • We clarify our right to talk publicly about aggregated statistics on Gimlet data — trends and the like.

We’ve actually done both of these things since early on in Gimlet’s life (see, for example, the question count on our homepage), but it’s nice to make this more formal.

As always, if you have questions about any of this, please let us know.

September 5th, 2014  |  Published in Uncategorized


Gimlet Turns Five

For years, I’ve been a huge fan of Pinboard. Like Gimlet, Pinboard has stayed simple, rather than adding every feature in the book. Like Gimlet, Pinboard has stayed small and profitable without chasing venture capital or seeking to be acquired for heaps of money. Like Gimlet, Pinboard is responsible for sobering amounts of people’s treasured data — a stable, durable island in the quicksand of internet businesses. And as I found out recently, like Gimlet, Pinboard recently turned five years old. The owner’s post is one of the best posts I’ve read on running an internet business. Seriously, read it. I’ll wait.

Still here? Well, in that vein: Here’s what the last five years have been like for me.

The biggest surprise in running a software startup has been the amount of non-code stuff I’ve done. Don’t get me wrong; I write a bunch of code, but when first dreaming about Gimlet, I imagined my time spent polishing a shining, flawless beacon of programming. Not so. Mostly, this is because Gimlet works and we don’t want to change things just for the sake of changing them. Partly, it’s because there’s a lot more pressing stuff that needs to get done: questions to answer, vendor forms to fill out (hoo boy do I hate vendor forms), books to balance, servers to update, the occasional retrospective blog post to write. And partly, it’s because our day jobs (yes, we have day jobs with health insurance and everything) still command a substantial slice of our attention.

(Also partly, it’s because Eric and I each had kids in the past few years. It turns out that kids are a crazy huge amount of work. But more on them in another post.)

Growth-wise, our clients are recording somewhere in the neighborhood of 22,000 questions on an average day. In some ways, that doesn’t seem like a huge number (it’s not even one per second!), but it means we’re on pace to add about six to seven million questions a year to the database, which isn’t nothing. We’ll cross the 10,000,000 mark sometime in the next month or so. It feels like just a few months ago that we were inching up on the 1,000,000 mark, and a few months before that we hit 100,000. For our library clients (which is basically all of you), it turns out that reference is by no means dead. Honestly, it isn’t even looking sick.

In another vague parallel to Pinboard, we’ve found that people will indeed pay for a good service, and that getting enough revenue to keep a business going is quite possible, as long as we don’t have illusions about gold-plated Ferraris or anything. Pricing is still something of a dark art for us (as is the case for most businesses like ours), but we’re pretty happy with our new pricing plan.

To avoid making promises we may not keep, I’m not going to talk about future feature plans. One recurring discovery for me has been that ideas that look good in my head and on paper are often terrible when I see them on the screen and get to click on them and try to explain them to people. Fortunately, there are enough ideas that seem good in our heads that some of them will hopefully look good on screen, too.

This has gotten rather long, so I’ll end with a final parallel to Pinboard’s post. We’re grateful — so grateful — to all of you who have joined us, stuck with us, and sent words of encouragement and feedback for the last five years. We’re both looking forward to serving Gimlet up to you for the next… five? Ten? Fifty? Well, many years.

July 16th, 2014  |  Published in Gimlet


On Security Certificates and their Challenges

Last week, after patching our servers against the Security Bug of the Month, we here at Gimlet got a new SSL Certificate, which your web browser uses to encrypt your traffic and verify that you’re sending it only to us. Unfortunately, this led to Firefox users being unable to connect for several hours.

One of the biggest dangers of the heartbleed bug is that it could allow Evildoers to download something called a server’s Private Key, and use it to do Enhanced Evil. In essence, your private key is a magic, randomly-chosen number that lets a a web site decrypt all the https traffic it receives. The only place that key should live is on the securing web server; operators never give it to anyone. Ever. Instead, operators use a program that does Clever Math to make a new number from the private key, and send it to a Certificate Authority who says “Ah, these people really are who they claim to be” and do yet more Clever Math and generate a Certificate. The operator then installs that on the web server and all is well.

However, with the private key, Evildoers could bypass the security provided by the certificate (using, you guessed it, Clever Math). In response, the right thing to do is get a new certificate from the Authority and have them tell everyone that the old certificate is bad. This process is called revoking a certificate. Where this gets tricky is telling everyone about the status of a certificate. In the past, every web browser was shipped with a list of revoked certificates; when there were a few thousand web sites, this worked fine. Now, there are millions and millions; maybe a third of them are going to revoke their certificates as part of this process. That would be one long list.

So instead, now, Certificate Authorities need to run something called an OCSP service. When your browser visits an https:// site, it’s supposed to contact the OCSP service and say “hey, is this certificate any good?” And it should say “Yes!” to good certificates and “No!” to revoked ones.

Our problem was that, after issuing our new certificate, our Authority’s OCSP service took several hours to start saying our certificate was legitimate — instead, it would respond “Unknown!” For added fun, every web browser treats OCSP responses differently. Chrome and Safari and Internet Explorer saw the Unknown response and decided it was good enough. Firefox decided that it was totally unacceptable. After about six hours, the OCSP server started returning a good response and all was well.

To make matters worse, the OCSP server decided to update its status for our old, revoked certificate within an hour or so. So, when we tried to roll back to our old certificate, Internet Explorer threw a “Revoked Certificate” error. Oddly, only IE seemed to think this was a big problem.

Going forward, if we ever do this again, we’ll make sure we have a certificate that’ll be recognized before we revoke the old one. This may mean switching Certificate Authorities, or just getting better service from the one we’re using now.

April 15th, 2014  |  Published in Gimlet


One other Heartbleed thing…

Last Wednesday, Gimlet’s own Nate Vack (that’s me!) was on Wisconsin Public Radio, talking about the Heartbleed bug. If you’d like to know more, have a listen.

April 14th, 2014  |  Published in Gimlet


Gimlet and the Heartbleed bug

As you may have heard, a vividly-named bug called Heartbleed was discovered in some of the internet’s most widely-used security software last week. It was discovered by security researchers and no one knows if evildoers used it to do evil; as a result, the messages from online services have been somewhat mixed.

Gimlet’s servers were running a vulnerable copy of the security software for about two years; we’ve been patched since Tuesday. In addition, we have updated the encryption certificates that let your web browser know it’s talking securely to our servers — we’ll post more on that (and its complications) soon. Finally, as the most likely thing an attacker could do would be to steal your Gimlet cookies and log in as you, we’ve cleared those cookies and made you log in again.

We don’t think you need to change your Gimlet password, and very much doubt that any sensitive information has been leaked. Please use the care you always use on the internet, and if you have any questions, don’t hesitate to contact us.

April 14th, 2014  |  Published in Gimlet


We’re Raising Our Prices!

What’s this all about?

We’re raising the price of Gimlet. Starting on September 1, 2014, the first branch for an account will be $20 per month; each additional branch will be $10 per month. More details will be up soon at our pricing page.

Wait, what? Why are you doing this?

Gimlet is our first product — indeed, our first foray into the business world. When we launched more than four years (and seven million questions!) ago, $10 per month was our honest best guess of what we’d need to charge to cover both ongoing support and continued development of Gimlet. We knew technical and billing support would take some time, but greatly underestimated it.

What we’re working to balance is this: We want Gimlet to be cheap enough that no one ever needs to keep reference stats by making marks on paper again. At the same time, we need to charge enough to build a sustainable business.

What about existing clients?

Existing clients may stay at their current price until September 1, 2014. While we’d like to make that a permanent offer, we can’t afford to.

At the same time, we don’t plan to make a habit of raising our prices. We know you have budgets and plans to prepare, and that changing prices complicates those. We know that the more data you have stored in Gimlet, the more it would hurt to migrate to another service — and we absolutely do not want to take advantage of our most steadfast clients.

What if we’ve paid ahead with an annual invoice?

If you’ve paid ahead, we’ll charge you the new price on your first invoice after September 1.

This makes us really sad! Or: We have questions!

Talk to us. Send us a message at support@gimlet.us, or email me directly at nate@gimlet.us. If you want a public discourse, post in the comments of this blog entry. If you’d like to talk over the phone, send us an email and we’ll set something up.

From us: Thank you.

We can’t express enough gratitude to everyone who has made our success possible by signing up for accounts, talking to us at conferences, sending us feedback, and telling their friends about us. We’re honored to be helping so many libraries do business.

February 24th, 2014  |  Published in Uncategorized


Of Databases and Character Encodings

Gimlet is old. Its roots date back to mid-2006, when it started life as a Ruby on Rails 1.1.6 program. Along the way, we’ve built up a bit of cruft here and there; last night, we cleaned a bunch of it out when we fixed the character encodings in our database. This paves the way for us to upgrade to newer and better tools.

Feeling like a nerd? Read on.

Hey what’s a character encoding?

Since the days of Braille, Morse code, and telegraphs, people have been using binary codes to represent letters and numbers. The idea is simple: As long as everyone agrees on the letters and numbers and their codes (say, ‘01000001’ means ‘A’) writing a program to turn those 1s and 0s into human-readable text is easy.

For many years, the “standard” character encoding was called ASCII (or the closely-related Latin-1) – which covered all the characters you see on a American keyboard plus a few extras. However! There’s no way to represent ב in Latin-1. Can’t do it. It’s not a Latin character. Because people really want to store documents in different languages, it was time for a new standard. After some fits and starts, most of the world settled on a standard called UTF-8. Instead of just being able to represent all the letters, numbers, and symbols used in American English, UTF-8 has enough capacity to represent all the letters, numbers, and symbols in every language written by humans.

As you might imagine, that’s the encoding we’re using in Gimlet.

OK so what did we do last night?

As we mentioned, Gimlet is old. Old enough that UTF-8 hadn’t completely taken over the world when we started working on it. But, being forward-thinking, we created the database with UTF-8-encoded tables and served our web pages with a UTF-8 encoding. Everything looked great – until we upgraded the software that connected to our database.

Suddenly, where we expected to see Gimlét, we saw Gimlét – and a variety of similar oddities. Upon closer investigation, it turned out that the old database connector had been talking to the database in Latin-1. We were taking UTF-8 data from the Web, encoding it as Latin-1, and storing it in UTF-8 tables. When we got it back out, the database connector kindly re-converted the data so it looked nice to everyone. The new database connector wasn’t having any of this funny business, so all non-Latin-1 characters looked broken to everyone. We asked our new database connector to deal in Latin-1, and the problem went away, though we were still storing “broken” data in our database. An uneasy stalemate was reached – until we went to upgrade Ruby.

Ruby 1.9 brooked none of this silliness. It steadfastly refused to cooperate in our decode-recode scheme, dutifully keeping the data in UTF-8 format all the way from the database to the screen, with predictably broken-looking results. People on forums suggested hacks, but they were ugly hacks. It was time to fix the data.

Fortunately, it was easy to see the problem in a standalone database management program:

select name from accounts where id = 1;

We needed to convert the text to UTF-8. Of course, the database already thought the text *was* in UTF-8, so this didn’t work:

select name, convert(name using utf8) from accounts where id = 1;
Gimlét   Gimlét

The trick was to do what our old database connector had been doing: take the data, interpret it as Latin-1, then re-interpret it as UTF-8. Because MySQL is “smart” about character encodings, we also had to tell it the characters were just a string of bytes after its Latin-1 encoding. Finally, we tell it to re-code the data as real UTF-8:

select name, convert(cast(convert(name using latin1) as binary) using utf8) from accounts where id = 1;
Gimlét   Gimlét

And from there, we wrote a little script to fix all the text columns in the database, and all was well.

July 30th, 2013  |  Published in Gimlet


Planned Gimlet outage: July 29, 11:00 PM-Midnight Central Time

As the title suggests, we’re planning some system updates the night of Monday, July 29. We’re planning to be down for about one hour from 11:00 PM to Midnight, Central Time.

July 24th, 2013  |  Published in Gimlet


One more exhibition musing

Next time you’re at a conference, look at the big booths. They’re paying between $15 and $25 per square foot for space. And $200-ish per hour (plus materials costs) to build their displays. And for all of the staff they have on-hand to talk to you and run informational sessions and whatnot. It’s pretty easy to come up with a ballpark figure on what that exhibition costs.

Interestingly, one vendor filed for Chapter 11 bankruptcy while exhibiting at possibly the biggest booth at the conference. Wow.

July 23rd, 2013  |  Published in Uncategorized