Remote-Url: https://zachholman.com/talk/utc-is-enough-for-everyone-right Retrieved-at: 2022-02-21 19:16:08.709567+00:00 UTC is enough for everyone ...right? ╲╱ Programming time, dates, timezones, recurring events, leap seconds... everything is pretty terrible. The common refrain in the industry is Just use UTC! Just use UTC! And that's correct... sort of. But if you're stuck building software that deals with time, there's so much more to consider. It's time... to talk about time. (Also it's time for a lot of time-related puns. Please prepare yourself accordingly.) Zach Holman / May 2018 / @holman clock hourglass-start watch alarm-clock stopwatch Since the dawn of time Years ago, I worked with a friend who had built a few scheduling calendars in a previous freelancing gig. Sometimes we’d be working on something that tangentially related to time, and as kind of a recurring in-joke he’d always tell me: Zach, whatever you do: just don't ever build a calendar. Anyway, I’m Zach Holman and I’m building a calendar. I guess I never could follow directions very well. Building a calendar sucks. Like there’s really cool shit you can do, since every calendar out there today is basically straight outta 2005, but at the end of the day you’re stuck dealing with all of the edge cases that all your dork friends have warned you about since the dawn of time. (Like literally, the dawn of time is a separate edge case you have to account for as well.) So there’s been a lot of heinous stuff we’ve had to work with. I’ve made notes of the egregious things I’ve learned about programming with time and pulled them together as a talk. This is the written companion piece, which is sort of a super set of the talk, since I had way more notes than what I could shove in a single talk. Beyond that, though, there’s also a lot about time itself that is properly hilarious, and it’d be a travesty to not talk about the country that recently decided to skip a certain day, or that the Unix epoch isn’t technically the number of seconds since January 1970, or that February 30 happened at least twice in history. It turns out humans have had a long, long history of poorly dealing with time, so when you hammer your head against the wall trying to deal with a timezone bug, well, you’re just the last in a long, long line of human beings that are terrible at all this! Congrats. You’re pretty unremarkable. lightbulb I've given this talk three times: at RubyConfIndia, RubyConf Australia, and Balkan Ruby. (Don't worry, non-Rubyists; there's no Ruby in this post. The conference topics were just happenstance. Also I love the word "happenstance".) Each of those talks were filmed, and I'll link to each of them above once the recording's been made available, in case you're more of a watch-and-listen kind of person. What is Time? What is time? What is time? Well, that’s a question as old as… time. But yeah, basically time is like, just a social construct, mannnnnnnnnn. 🌱💨 Physicists are still debating on whether or not time actually exists in the universe. This quickly gets us into the territory of asking ourselves really weird things that hurts our brain, like what is time, or what is physics, or if you only eat one donut from Krispy Kreme, did you even actually go there? So to avoid all of this, I’m just going to say that no, time doesn’t exist, therefor writing code for it doesn’t matter, so boom, we’re done here! Thanks for reading. Okay you’re not getting out of this that easy. But I think it is worthwhile to take a quick look at the history of time so we can figure out why it’s so goddamn hard to do things with timestamps. lightbulb This stuff gets wild to read more about, in the existential what-even-is-anything-anyway? category of questions in life. Apparently time gets more fuzzy as clocks get more and more accurate. Pretty sure this confirms we're living in a simulation, and the programmers were too lazy to fully flesh the code out before they reached their ship deadline. A brief timeline of time 1. 4000+ BC Astronomical clocks We start seeing a bunch of "clocks" pop up. But like, they were shitty clocks: basically they'd be like the equivalent of looking up, seeing a sun, and like boom, I guess it's daytime. 2. 1500 BC Sundial subdivisions Oh rad, now we start getting some nifty stuff. We start seeing more sundials split up into subdivisions, so we can be more specific about how far through the day we are. Egypt was the first ones to really start doing this: they had a duodecimal system already, so that's why it was split as base twelve, or our two parts of twelve hours in a day. If you're like me and immediately said ohhhhhhhhhhhh, so THAT'S why there's twelve hours in a day! and immediately followed it up with: wait, why the fuck are they using twelve instead of ten? Base ten is lit and doing everything in twelve feels so uncivilized, well good use of "lit" there. It's most likely because you have twelve joints in your hands: three in each of the four fingers, excluding the thumb. I thought that was pretty nifty to discover. Like, I had never looked at my hands before, really. Hands are really wild, when you think about it. 🌱💨 3. 14th century Fixed-length hours European clocks started standardizing on fixed-length hours; before, a lot of clocks would change the duration of an hour depending on the season. Basically anything with time that seems weird like this can probably be summed up with "the goddamn farmers did it", since, you know, growing seasons were really important to figure out when you couldn't just walk down the block and go to a Taco Bell Cantina® for a Baja Blast™. 4. 16th century Minutes Wasn't until around the 16th century before we started seeing minutes. Before then, nobody really cared, and mechanical clocks weren't good enough to count 'em accurately, anyway. Probably around this time your first boss's ancestors were also discovering minutes as a good way to make sure your ancestors got to work on time: "You're exactly 56 minutes late today, Holman, what the fuck is wrong with you? Go shave the sheep!" 5. 17th century Seconds Did you hear about the clock maker who was the first to add a second hand to a clock? His first prototype was a complete failure, but he got it working the second time. 6. 1883 USA standardizes time We start seeing some actual standardization on time itself. In what is the most bourgeoisie example of the most bourgeoisie era, a bunch of rich, white railroad tycoons met at a fancy Chicago hotel to agree on a standard timezone so their trains would work better together. They used the new-fangled telegraph to synchronize time signals between cities. Other countries would do the same thing around this era. Previously, it was basically a free for all as to what time it would be in any given city: Lamenting over timezone programming led me to this 1857 chart, and suddenly timezones seem GREAT. (At noon in DC, it was 12:08 in Philly!) pic.twitter.com/mSYyUKL6lX — Tess Rinearson (@_tessr) August 25, 2017 So, consider yourself playing in easy mode today. You can't imagine how much harder it would be to be a JavaScript programmer in 1882. 7. 1967 Atomic standard The world started standardizing on what time measurements actually are. If you have a pile of Cesium atoms sitting in your backyard, go out sometime and measure 9,192,631,770 energy transitions of a single atom of Cesium: because that is exactly the duration of a second, settled upon in 1967 by a bunch of scientists. Nowadays, our atomic clocks are accurate to losing a single second over the course of a few hundred million years. This is something like only losing two nanoseconds of accuracy a month, which definitely accounts for why I'm always late to your meetings on Monday morning. The clocks being worked on today, though, are even better: optical clocks "tick" about a hundred thousand times faster than atomic clocks, which can lead to way higher accuracy. Basically if you somehow started one of these bad boys right at the Big Bang about 14 billion years ago and somehow kept it going, it would have only lost a few seconds of accuracy. 8. 2018 You fuck up yet another timezone bug You didn't think I'd forget about you, did you? clock hourglass-start watch alarm-clock stopwatch Everything we just covered revolves around a single question: What time is it? At a certain point we started asking this question. This became important to know when you could finally head home after work, for example (answer: never early enough). And for a long time that was fine, but that wasn’t entirely the question we were asking. There was a hidden implicit part of the question we were never asking: What time is it here? Once we started getting planes and trains and automobiles, we had movement, we had transportation, we could be in two distant places within hours. So we started asking this question: What time is it there? This question ruined everything. Now we had to deal not just with one time (here), not two times (here and there), but a multitude of times, all interacting with each other (everywhere). This complicated everything, and made obvious the notion that writing timezone code was some of the worst things you have to do in our field. It’s so predictable that developers will pooh-pooh having to write timezone code, almost as much as it is predictable that some clueless commenter on Hacker News will complain that this page has autoplaying video on it. And then someone will calmly quote this passage in response, quietly pleased with themselves that the initial commenter was rude and certainly didn’t read the post at all. Then a third person will chime in on the thread saying the author was playing you all like a fiddle anyway, and the real problem is that the post was way too long to start with. [757px-Univ] This is terrifying. It's a "Universal Dial Plate", showing times of individual cities of the world, from 1854. Gulp. Timezones So you’re starting out fresh, building a completely new global time structure. Clearly, since you’re a programmer, you already have an innate distrust of timezones, so you’re going to get rid of them entirely, a la Swatch Internet Time, which demonstrates why you are the beautiful human you are, you dashing human you. In fact, the title of this post is wrong: UTC isn’t enough for everyone; you should get rid of UTC and move all your servers to use Swatch Internet Time instead. Okay, now that we got that tomfoolery out of the way, we start asking questions about what time should look like. And if we do have timezones, the obvious thing to you is to split the world up into twenty four timezones, one for each hour in the day, so it’s all very neat and orderly. Spoiler alert! Nothing is neat and orderly when it comes to time. There’s something like 39 timezones at time of writing. This, like all of the problems with time, is political. Timezones get added gradually, over time, depending on political whims, geographical reality, and economic plausibility (more on a lot of these later). So, as a common theme you’ll see, this is just one example where we try to graft an irregular system (time and timezones) over a regular system (the duration from time point A to time point B). This stuff moves a lot, too: the tz database (also known as the Olson database), which is the listing of timezone rules we use as programmers to calm this chaos, gets updated many times a year. It’s also worthwhile to point out here that timezones aren’t tied to hours: many timezone offsets happen at 30 minutes, 45 minutes, and so on. So we need to stop thinking purely about “hours” and start thinking about offsets, in a way. Timezones are weird. Time is weird. Why Time is so hard Who needs December 30, 2011 anyway Let me tell you a quick story. Do you remember what you were doing on December 30, 2011? You have three seconds. Go. Nope? Nothing? Well then, if you don’t know what you were doing December 30, 2011, then there’s one obvious explanation for this: you must be Samoan. You see, Samoa didn’t have December 30, 2011. They went straight from December 29 to December 31, do not pass Go, do not collect $200. Samoa used to be east of the International Date Line. I mean, it still is (they didn’t physically move the country, which would be a pretty impressive feat, now that I think of it), but it used to actually follow the date line accordingly. In the years before 2011, Samoa looked up and realized that hey, most of our major trading partners — China, Australia, New Zealand, and the like — are on the other side of the date line, so when we’re dutifully working away at the office on Friday, the rest of the people we actually do business with have already fucked off for the weekend. The Kiwis are probably off grilling meat somewhere, never to be heard from until Monday, if we’re lucky to hear from them ever again at all. So Samoa decided to change timezones so that they could line up better economically. It’s also important to note that Samoa is about 100km away from American Samoa. Not super far at all. Samoa and American Samoa map So at this point in my talk I say the next part real fast, so like, just read it real fast and we’ll try to approximate me. It’s best if you do a pile of cocaine before reading the next part. I’ll wait. Okay, so: At the start, Samoa and American Samoa were in the same timezone, the Samoa Time Zone, which encompasses the two different countries, except it’s actually one country and one unincorporated territory of the United States (American Samoa), so when Samoa pulled out of this time zone it switched to the West Samoa Time Zone which was UTC +12:00 compared to Samoa Time Zone which stayed at UTC -11:00 which meant West Samoa Time Zone was 24 hours ahead of Samoa Time Zone which isn’t entirely accurate because sometimes it would be 25 hours ahead of Samoa Time Zone because Samoa Time Zone doesn’t observe Daylight Saving Time and, of course, West Samoa Time Zone also doesn’t observe Daylight Saving Time except it does observe it starting in 2009, wait that’s not true they had to push that back to 2010 because of the Samoa Earthquake and Tsunami which messed everything up; now naturally all of this may sound familiar to you because this is the mirror reverse of what happened on July 4, 1892 when the opposite happened when they looked around and said hey our major trading partner is the United States, why are we in a totally different day than these Yanks, so let’s do July 4, 1892 twice (which is doubly awesome and hella 🇺🇸 🎆 P A T R I O T I C 🎇 🗽, since it’s Independence Day in the USA); it’s also important to note that Tokelau, a nearby territory of New Zealand, saw all this happening so they got in on the action and did the same thing and moved to West Samoa Time also on December 30, 2011. When all of this happened, the BBC sent a journalist to Samoa to see how they were taking it: You can hear the sound of a lot of vehicles going round town, going round the town centre clock and tooting their horns. People screaming. “We have once again achieved another milestone in our history of Samoa.” I don’t know who they interviewed for this piece, but what I do know is whoever said this thing never had to fucking deal with fucking timezone exceptions in code before. Fuck. Working with time is hard. There can be little edge cases like this every step of the way. UPDATE: September 23, 2021 Here it is, three years later since I originally wrote this, and the journey is still ever-changing. Samoa just announced that they’re getting rid of Daylight Saving Time. They were supposed to switch over in three days, but now… they’re just… not. Time is great! clock hourglass-start watch alarm-clock stopwatch No Russian The Russian Olympic shooting team, pleased to represent their country on The Big Stage, was proud to head to London for the Games in 1908. They hit the road… and made it to London twelve days too late and missed their event. Turns out, in 1918, Russia switched to the Gregorian calendar. This was neat, because that moved yet another country to using Gregorian, so we could all be on the same page. In 1908, though, they were still on the Julian calendar. England, the host of the Olympics, was on the Gregorian calendar. So when they told them to show up at a certain date, the two calendars didn’t match up. Whoops. This mix-up was a bummer, but possibly not too unfamiliar for Russia, as the same thing happened in 1805 during the Battle of Austerlitz. Napoleon was doing his thing and invading Austria, and the Austrians, assured that Russian reinforcements would show up, you know, in time, was sad to also discover the woes of different calendars as reinforcements showed up twelve days too late. Napoleon ended up winning. clock hourglass-start watch alarm-clock stopwatch Other interesting factoids that I could expand upon, you know, if only I had the time: • February 30 happened at least twice in history. • An Earth “day” is different from the time that it takes for the Earth to rotate on its axis once. This is because the Earth is also rotating around the sun at the same time, so that measurement is actually something like 361° of rotation instead of the normal 360°. • You ever forget if “12am” means noon or midnight? I always felt stupid having to think about this every time, but I feel less bad after reading this excellent Stack Exchange answer that details the entire history of this. My favorite part: the United States government decreed that “12am” meant “noon”… up until 2008, when the United States Government Printing Office reversed their position and swapped to using “12pm” as “noon”. Time is hard, and you’re not the only one screwing all this stuff up. Code & Time While I was initially building this talk, I found myself listening to the words of my absolute favorite modern-day philosopher: Nas. You got CD-ROM, everythang operates by computers Then what happens when circuit breaks? Y2K’s the big scare Scientists say we ain’t prepared - Nas, "New World", 1999 I’m not really sure what to take from all this, other than if Nas is really concerned about computer science and how it relates to time, then dammit, we should be concerned, too. Nasty Nas Storing Time The very first thing you need to start worrying about when you build smart, time-aware software is how you store time. For many, databases will be the tool they’ll reach for. And there’s a lot of things to consider when it comes to how you store that data in a database. Step one: use UTC. Okay I’m not going to suddenly say all this advice we’ve been giving for years and years is wrong. UTC is a fine standard to base all your times off of. So use it. Don’t do something silly and change your servers’ timezones from UTC. Now, I’ve been burying the lede here for quite some time. It’s worthwhile to talk quickly about UTC, and why it’s a good default when we tell developers “Just use UTC”. UTC, of course, stands for:         Universal CoordinaTed…?        [DEL: C :DEL] um… lemme figure this out… Time? Time standards bodies got us making scratch pads like some Zodiac Killer letter. Didn’t even know Ted Cruz cared about timezone issues. But yeah, this UTC initialism doesn’t make any sense. Let’s dig into this a little more. So you’ve got a bunch of scientist types around 1960 who are like, hey, time is all screwy we should totes make a standard. And some of them spoke English, and some of them spoke French, which, of course, is the cause of so much conflict over so many generations. (In hindsight, maybe we should have split all those troublemakers up from the start.) The English-speaking folk were like yo, this definitely sounds like Coordinated Universal Time, boom, ship it. And the French speakers were like yeah that makes total sense! Temps Universel Coordonné DOES work out well in our language, too, ship it! Then they both looked up and realized cool, they’ve created both CUT and TUC for acronyms. Shit. When your standard — that is expressly meant to standardize time — doesn’t even standardize on a standard acronym, well, damn, that probably doesn’t bode well for your standard. Or, actually, now that I think about it, this might be the most perfect metaphor about time itself ever created. Maybe we should give them some credit for that. Time never makes sense. Anyway, the compromise that arose was that if everyone is special, no one is special, so they created an entirely new set of letters that has no direct relation to any real words for the compromise: UTC. Whew. So. We use UTC because it has an offset of 00:00: in other words, it has no timezone offset. Other timezones are offset from UTC, not the other way around. It’s important to note that UTC is not GMT. GMT is Greenwich Mean Time, which historically was the center of time in the world. They both have an offset of 00:00. But you shouldn’t use GMT on your server, for example, because UTC is a standard, but GMT is a timezone. People actually live in locations whose time is GMT; UTC isn’t directly used by people (unless they’re really weird). This may sound like such a small difference of wankery, but the more relevant part is that some users of GMT observe Daylight Saving Time (but not all). UTC has no concept of Daylight Saving Time at all. So if you base everything in GMT, well, who knowssss what will happen. lightbulb Quick side note: it's Daylight Saving Time, not Daylight Savings Time. You don't want to sound like a total tool in front of all six of us who care about this stuff. One question you might want to ask right from the start is: do you really need to store time? As programmers, we’re kind of inherently built to want the ABSOLUTE BEST HIGHEST FIDELITY FORMATS OF ALL TIME. Like dammit, I need the timestamp down to the micromillinanosecond for every cheeseburger that gets added to my bespoke Watch-The-BK-Throne app. If I do not have this exact knowledge to the millisecond of when I consumed this BBQ Bacon WHOPPER® Sandwich From Burger King® I may die. But we sometimes don’t need all this data, and that’s cool, so be sure to ask yourself if you really need this data, or you just want this data. The rule of the game for programming with time is that you should opt to go as simple as possible as soon as possible. A good example of this are birthdays. Birthdays are kind of like floating events; it doesn’t really matter where you are on the planet; if the numerical day matches up with what’s on your driver’s license we’re set. Storing these things as a timestamp column instead of just a date column in your database can end up complicating your code down the line. Properly storing timezone-aware times On the other hand, you may actually want to track time. In those cases, there’s a few ways of doing this, particularly if you know that fidelity of hour or minute is going to be important for your use case. Say you have an events table in your database, and you toss in a column to keep track of when the event starts: Name Type starts_at timestamp This is neat and obvious, since you want to store a time as a timestamp in whatever database you’re using. But if it’s important that you don’t end up being an hour off when something actually happened, you’d need to store the originating timezone as well: Name Type starts_at timestamp starts_at_tz string By adding the offset as a string in your database, you gain some additional powers. For one, it gives you a closer idea of when specifically the event happened. One way to do this is to treat this column as an integer; say, store -240 in a row to represent a shift of 4 hours (4*60 minutes). That’s cool, and it does get you closer, but again, we’re talking about a use case where it actually does matter to be as accurate as possible (and being off an hour might lead to nonsensical data). Just having a numerical offset doesn’t give you fidelity of location. Was it during Daylight Saving Time? What about comparing that to today’s time; do we account for DST or not? Also, do we need to account for changes in timezones that might have happened since that point in time? Instead of a numeric offset, use a string. Specifically, use the full qualified name in the Olson database, so something like Australia/Adelaide or America/ Los_Angeles. These are standardized descriptors of the timezones used in the world, and you can use these in pretty much every programming language ever used in the last few decades. Again, all of this probably isn’t relevant for every app — who cares if someone’s browsing the article Brad Pitt’s Beautifully Coiffed Hair: Friend or Foe to the Proletariat? and my six comments are technically listed as an hour earlier than they actually were. Regardless of timing, my comments were perfectly relevant and explored the issue perfectly. But for some apps it might make sense to get this detailed, particularly if you’re comparing a lot of times to each other (like, say, a calendar). lightbulb The historical side of this stuff gets kinda cool, too. One of my favorite examples is in RFC 3339: 1937-01-01T12:00:27.87+00:20 This represents the same instant of time as noon, January 1, 1937, Netherlands time. Standard time in the Netherlands was exactly 19 minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through 1937-06-30. This time zone cannot be represented exactly using the HH:MM format, and this timestamp uses the closest representable UTC offset. So if you really get down into the rabbit hole and find yourself needing historically-accurate times across decades, you might need to backtrack over multiple previous timezone changes in order to get to that point. Also if you find yourself at this critical juncture, you might want to just give up programming entirely, because that sounds horrible. Transiting Time Aight, cool, you’re storing your times correctly; now we can take a look at how we transit those times to our clients. Number one rule: stay consistent. The number one way to address that number one rule: ISO 8601. ISO 8601 is one of my favorite standards and/or RFC out there. And yes, you should definitely have a favorite. (My all-time fave is RFC 2606, thanks for asking! I’m in awe of that absolute unit. Where would we be without that banger? We’d be in complete fucking chaos, that’s where.) ISO 8601 was a standard that came out in 1988, so it’s probably older than all you lot who are reading this, now get off my damn Friendster lawn. It was re-upped in 2004, 2014, and is anticipating a new draft by the end of this year. It basically defines THE way of writing a timestamp. Here: I’ll even be nice and show you an example: In JavaScript land, this is simple: it just uses the toISOString() function: new Date().toISOString() The astute among you might see a few reasons right away why this is such a dope format: • Easy sorting: It arranges all components from large to small, so pretty much any novice in a programming language could easily sort a list of timestamps from latest to oldest. • Timezone information: At the far end it includes the offset: -08:00, for example (well, not in this above example specifically, since in JavaScript you’d also need to parse this out manually with getTimezoneOffset()). This isn’t as high-fidelity as when we were talking earlier about storing the qualified string version of a timezone, but it does give us information we wouldn’t have otherwise. You don’t get this if you pass around UNIX epoch time, for example. • No locale problems: You don’t have problems with Month/Day/Year formats getting confused with Day/Month/Year formatted dates. More on this in a few. So yeah, use ISO 8601 over the wire. It’s the most popular way of tackling a shared standard for time, so don’t be the jerk who does it in some fancypants custom manner. RFC 3339 Wanted to mention RFC 3339 briefly as well, since sometimes you’ll see its mean mug pop up in APIs and other spots from time to time. RFC 3339 came out in 2002. This is a simplification, but it’s kind of a subset of ISO 8601. It requires timestamps — ISO 8601 allows you to omit them and just use it for a date — and in general it’s slightly more strict. It lets you get away with less. Effectively they’re sort of interchangeable. Most people and API platforms just say they comply with ISO 8601, so that’s probably going to be fine for you, too. xkcd on ISO 8601 Displaying Time British press corp is locked outside the White House because our birthdates were submitted in UK format and secret service don't get it. 🇬🇧 — Jim Waterson (@jimwaterson) January 27, 2017 You’ve got some time, you’ve sent it to the client or to the user… now you have to display it, with hopefully minimal amount of confusion with the United States Secret Service. I don’t know if you’ve noticed this yet, but Americans aren’t perfect. We use this fucked up date format, MM/DD/YYYY. Like under no circumstances does this make any sense. But if you switch that around and try to show someone from the US a date like 24/6/18… well that’s just going to be weird. People like their locale-based date formatting, and you should try to respect that as much as you can. One of the ways you can do this on the web is using the Intl API. Intl supports a number of different hooks and abilities to handle locale- and language-based changes for numbers, plurals, and dates and times. Beyond that, if you’re going to be doing anything fancier with dates and times beyond, say, just printing something to the page, you should really start using a proper time library. Things like adding two dates together while still tracking DST and a multitude of other time rules… well, it’s all really difficult to do yourself. Stand on the shoulders of giants, and steal from someone else. Just how Open Source was intended. Some solid libraries to take a gander at: • moment.js The classic time library in JavaScript. Date manipulation, formatting, pretty much everything you’d need. • date-fns More modern approach to a moment.js-like experience for handling dates on the web. • github/time-elements Web component extension to the