What. A. Week. I've been absolutely non-stop publishing data breaches to HIBP whilst simultaneously putting in place the framework to start advising NordVPN on their cybers and open sourcing the HIBP code base at the same time (and a bunch of other more boring stuff that didn't make the cut). That's all explained in this week's update so I won't drill further into it here, there's obviously a couple of big announcements so if you have any questions, drop them in the comments below and I'll either them there or take them up in next week's update.
Let me just cut straight to it: I'm going to open source the Have I Been Pwned code base. The decision has been a while coming and it took a failed M&A process to get here, but the code will be turned over to the public for the betterment of the project and frankly, for the betterment of everyone who uses it. Let me explain why and how.
HIBP is a Community Project
I've been giving a great deal of thought to how I want this project to evolve lately, especially in the wake of the M&A process that ended earlier this year right back where I'd started: with me being solely responsible for everything. The single most important objective of that process was to seek a more sustainable future for HIBP and that desire hasn't changed; the project cannot be solely dependent on me. Yet that's where we are today and if I disappear, HIBP quickly withers and dies.
As I've given further thought to the future since the M&A process, the significance of community contributions has really hit home. Every single byte of data that's been loaded into the system in recent years has come from someone who freely offered it in order to improve the security landscape for everyone. Many of the services that HIBP runs on are provided free by the likes of Cloudflare. Much of the code that's been written has drawn on community contributions either by virtue of content people have published publicly or support that's been provided to me directly.
I was reminded of this just yesterday when my friend from Cloudflare, Junade Ali, posted this:
This tweet isn't entirely accurate; it was all Junade's idea and he designed the k-anonymity implementation for HIBP's Pwned Passwords. For free, because he's a good bloke and Cloudflare supported him. LastPass has now employed that same model and they follow the other notable names Junade mentioned. I'm sure I speak for him as well when I say we couldn't be happier that other companies have taken the model we pioneered and applied it to their own services too because at the end of the day, that's in everyone's best interests.
The philosophy of HIBP has always been to support the community, now I want the community to help support HIBP.
Open sourcing the code base is the most obvious way to do this. It takes the nuts and bolts of HIBP and puts them in the hands of people who can help sustain the service regardless of what happens to me. But this isn't just a philosophical decision based on a desire to offload work, it's also common sense for a number of reasons. Let me explain:
The point is that it was always the intention to be completely open about the design of HIBP, it's not like there's any proprietary secret sauce I've been trying to protect here.
Open Source is Everywhere
A heap of really amazing projects are open source these days. Visual Studio Code, for example, is open source. The platform this very blog runs on, Ghost, is open source. Most of the libraries HIBP uses are open source. And I'm not just talking open source in the "source open" kind of way where other people are free to read it, but I'm talking open source in terms of taking contributions as well.
It actually got me thinking - how many of the products and services I use every day are open source? I asked on Twitter earlier today, and it's, well, extensive:
Putting the code out there in public goes a long way to addressing concerns people have about the way the service operates. For example, people have often questioned whether I'm logging searches in order to build up a new list of email addresses. No, I'm not, but at present that assertion effectively just boils down to "trust me". Showing the code - the actual code - and demonstrating that things aren't logged is a very different proposition.
Open sourcing the HIBP code base gives me the opportunity to address that original problem I set out to solve with the M&A process: finding other people that can help sustain the project. All that backlog, all those bugs, all the great new ideas people have but I simply can't implement myself can, if the community is willing, finally be contributed back into the project.
And that's something that I'm adamant about; the goal here isn't just to say "hey, look at the code, it's not logging your searches", it's fundamentally about making HIBP a more sustainable, more robustly featured community service. Frankly, I can't think of a single good reason why I wouldn't do this. But that said, it's also not as trivial as it sounds so let me talk about the practicalities of the whole thing.
Practically, There's Work to be Done
I started writing HIBP on a plane to the Philippines in 2013 and finished up a bunch of it in a hotel room once I landed. In the near 7 years since then, I've chipped away at it in little bits and pieces, frequently from a laptop while travelling, jet lagged and preoccupied. I've taken shortcuts. I've hacked together some pretty messy stuff. I've probably checked in secrets before and when you're the only person touching a project you can get away with all that stuff, but not once you start opening up source.
HIBP isn't in a state to simply flick the visibility of it in GitHub, but it needs to get to that point. Instead, I need to choose the right parts of the project to open up in the right way at the right time. That exercise alone requires help and for a while now, I've been talking to some of the smartest people I know in this space. People who live and breathe open source, people who understand .NET and Azure inside and out, people who know HIBP well and above all, people I trust to expose my own shortcomings so that they can help me make this thing more sustainable. With their support, the transition from completely closed to completely open will happen incrementally, bit by bit and in a fashion that's both manageable and responsible. Let me be clear: I don't have a timeline for each step along the way yet as HIBP remains something I do in my spare time and I've always got a bunch of other stuff on my plate, but the process has already begun and I'll be sharing more on that as soon as I can.
I want to get to a point where everything possible is open. I want the infrastructure configuration to be open too and I want the whole thing to be self-sustaining by the community such that I make myself redundant. That's not to say I'm planning an exit (far from it), but it's not good for HIBP that I can't exit right now and frankly, it's not good for me either.
The point is that the goals outlined in this blog post will take time to reach and they're not as trivial as they may sound at face value. HIBP remains a pet project run when I have the chance and somewhere within there I need to make the commitment to get it to the point I'm aiming for in this blog post.
What About the Data?
I need to really clearly break this part of the discussion out because whilst open sourcing the code base is one thing, how the data is handled is quite another. There's no way to sugar coat this so I'll just lay it out bluntly: HIBP only exists due to a whole bunch of criminal activity resulting in data that's ultimately ended up in my possession. Of course, the situation is a bit more nuanced than that with the vast bulk of data in HIBP already being in broad public circulation and passing through many hands. But be that as it may, even the legality of possessing it remains grey and whilst there are many internet armchair experts chiming in with their own opinions on the topic, here's what the legal guidance I've consistently been given boils down to:
We invite parties to form their own views on the legality of the data
Great, nice lawyer speak there guys. (And seriously, yes, that's what the KPMG lawyers from the M&A process I paid an eye-watering amount to advised.) Yet clearly, many of the world's largest companies do see value in it and conclude that holding the data is acceptable. Big tech companies, for example, pull down precisely the same breaches that go into HIBP and use them to identify credential reuse across their own platforms:
Then there's the privacy side of it all: my own personal data is in those breaches and your data almost certainly is too because there are literally billions of people that have been impacted by data breaches. Regardless of how broadly that information is circling, I still need to ensure the same privacy controls prevail across the breach data itself even as the code base becomes more transparent. That's non-trivial. Doable, but non-trivial.
This is something I've given a lot of thought to for a long time now. The concept of open sourcing HIBP has been floated over and over again and it's taken a failed M&A process to help me realise that this was the best path forward, but now here we are.
I've used the word "community" a lot throughout this post and I can't understate the importance of the role other people have played in the project's success. Just to really drive that point home, look at how many breaches have gone into HIBP in the last two weeks. At the time of writing, that's 16 breaches encompassing 95,850,490 records and every single one of those has been a community contribution; someone selflessly standing up and trusting me to handle the data in the best interest of others. I focus on that short time frame in particular here because it also demonstrates the constant flood of data and the need to scale myself more efficiently.
So that's where HIBP is heading. I know this blog post will be met with much enthusiasm because that's what many of you have been telling me to do for a long time. I've listened, now it's time to make it a reality 😊
I love security. I love privacy. Consequently, it will come as no surprise that I love tools that help people achieve those objectives. Equally, I have no patience for false promises, and I've been very vocal about my feelings there:
But one of them is literally called “Secure VPN”, how is this possible?!
“Are You Using These VPN Apps? Personal Info Of 20 Million Users Leaked: That’s 1.2TB Data” https://t.co/BPDww70Pgo
VPNs are a great example of where a tool can be used to enhance security and privacy but often, they fall short of delivering on the promise. When you use a VPN, you're trusting a third party with your traffic and even in an increasingly "encrypted by default" web, you're taking a leap of faith with who you choose to route your bytes.
A few months ago, NordVPN sponsored this blog and we got to chatting. I had a long call with Tom Okman (that link is a good read on their background) who co-founded the company in 2012 and I expressed my dismay at the trustworthiness (or lack thereof) of so many VPNs in the market. This was before the embedded tweet above but well after I'd written about dodgy VPNs:
Whoever can see your traffic - be that your local ISP or the VPN provider you decide to use - has an enormous responsibility and you're placing a huge amount of trust in them
I really pressed Tom on the trust piece - why should people trust NordVPN? The promise of "no logs" in particular is a favourite of VPN providers yet evidently, the reality doesn't always meet the promise. Turns out they'd just had their second PWC audit to verify their claims and came out clean which is a pretty solid way of demonstrating their commitment to privacy. Having a Big Four do any sort of formal audit wouldn't have been a cheap experience and the fact Tom and co recognised the value, not just in making claims but proving them too, carries a lot of weight.
But there were also aspects of NordVPN I told Tom needed work, especially around their messaging in marketing material. Look, I get it, marketing people like to embellish but, in my view, there were occasions where that went beyond what you could reasonably expect a VPN to do. You can't on the one hand put all this work into trust and transparency and then on the other hand convey messaging that impacts trust and transparency! And yes, I have strong views on these things 😊
So Tom asked me if I'd like to become an adviser to NordVPN and invest a bit more time than just a telephone call sharing these ideas. I thought about it for a while, kept using the product, liked it, realised it's not like I'm travelling anywhere anytime soon so I've got the time and gave him a thumbs up. So here we are. I'll be devoting some cycles each month to work with NordVPN on their tools and messaging with a view to helping them make a great product even better. Yes, it's a commercial relationship but no, I won't be employed by them, will remain independent and will continue to do all the things I usually do anyway (except travel, of course).
NordVPN has done a great job getting their product out to 14 million people worldwide and frankly, that's a pretty impressive number for a tool your average consumer has no idea about. I'm looking forward to working with them on the product, reaching more people and having a greater positive impact on digital privacy.
Unfortunately, our run of good luck here down in Aus has taken a bit of a turn COVID wise. Not so much in my home state, but the southern states have been copping it so this week, I pulled the pin on snowboarding. For folks overseas, that might sound like it would have been a risky proposition anyway, but only two and a half weeks ago the entire state of New South Wales had 5 active cases out of 8.1M people. Today it's 209. It's neighbouring southern state of Victoria had 40 cases in the middle of last month. Today they hit 5,385. It was just too risky for something that ultimately, is a luxury so I'm staying put in Queensland where we've had our own uptick, but so far it's only from 2 active cases earlier last week to 11 as of today. So far.
Back on business as usual, I've been processing a ton of breaches lately as the ShinyHunters data in particular has hit the public airwaves. There's a heap more to go yet too and I'm not even half way through so stay tuned to a lot more pwn notifications coming your way (sorry!) I also spend a bunch of time today talking about how and why I specced my version 2.0 office the way I did, especially given the way some of the commentary on Hacker News was worded. I found that kinda amusing, I hope you do too 😊
So many breaches... (that's a link through to HIBP's Twitter timeline, check out the last week)
I love this setup! A huge amount of research went into this but the PC, screens, cameras lights and all the other bits are working really well together. I did my first interview with this setup today and I think I'm actually going to be sticking with the mood lighting for most on-video events now:
I was searching around for a quote along the lines of you only being as good as the tools you use and somehow, I ended up down this rabbit hole of painters quotes and carpenters quotes and stuff about artists and their brushes. Then I started thinking it sounds a bit obnoxious anyway so maybe it wasn't really that relevant (yet somehow, here we are...) until eventually, I thought "stuff it, let's just write about the computer bits". So here we are.
More than a decade ago, I wrote about building the ultimate home office. Truth be told, that was originally written as an internal blog post at my old corporate job, largely because I wanted to demonstrate that a lot of thought had gone into my home workspace and consequently, I should spend more time working from there instead of from my windowless corporate office. How times have changed!
Moving forward, last month I started suffering a series of equipment failures. First it was my trusty Electro-Voice RE320 from 5 years ago that saw me through so many Pluralsight courses, webcasts, interviews, etc. Dead. Cactus. No sign of life whatsoever. Then the PC itself started sounding really unhappy, like the bearings in a fan had gone. No problem, I'll just replace the fan except... I couldn't identify which one it was. Eventually I worked out it was actually a pump in the water-cooling system and shortly after, I started getting thermal shutdowns. I'd had a bunch of issues with the BIOS losing settings on reboot too so a long time back, I actually snapped a few pics of the setup so I'd never forget it. How old was that again?
Alright, almost 8 years of life for a PC that's rarely ever even been turned off, that's a good innings! Time to rebuild from scratch, here's what I've done:
I'll start with the easy bit because none of this has changed:
I commented in the original post that a good chair was well worth the spend and I'm still sitting on the same Herman Miller Aeron as I type this. I've replaced some foam and that's it; this chair is probably the single best investment I've made in my workspace and if I write another post a decade from now, I expect I'll still be sitting on it.
The desk is the same one too and I find it just the right size to fit everything I need on the main part (which is precisely 2m wide), and secondary stuff such as a laptop and iPad on the return you see on the right. It's a solid, sturdy unit and just like the chair, it'll last another decade yet too. Loads of people have said they love the desk and asked where I got it, it came from Domayne in Australia but obviously more than a decade ago now and I can't see it on their current catalogue. If you want to do some sleuthing, it's featured in this office style profile page and I believe it's called the "Loren" desk.
This was painful, primarily because there were so many responses when I reached out and asked people for guidance:
Often, a particular product stands out above all others when I ask a question like this (for example, Ubiquiti when I rebuilt my home network years ago), but not this time. Although there was a trend - people love RØDE. This was pleasing because firstly, all the feedback was very positive regardless of the model and secondly, it's an Aussie brand (regardless of the funny letter in the name) and particularly in this dire economic time, I love the idea of supporting local. Problem is, now I've got the paradox of choice where there are so many possible options I'm paralysed from actually doing anything whilst I assess them all! Eventually I reached out to someone at the company and they made this recommendation:
Based on all the information you provided Adam, I’d recommend checking out the RØDE Broadcaster. The Broadcaster is a large diaphragm, end-address condenser mic that’s a standard in radio stations all over the world. Although it is a condenser mic, it sounds like you’ve got a quiet enough room with good enough acoustics that it won’t matter. Combined with the WS2 pop shield, you’ll have a fantastic sounding microphone that will make your voice sound superb.
So that's it mic wise, the only other sound-related thing I'm tempted to do here is to put some soundproofing on the wall in front of me. It's a stark white wall at present with nothing on it and I'm thinking of even making it a bit of a feature. We'll see.
Alrighty, the big one. I have a really acute sense of when something is killing my productivity; interruptions, external noises and probably more than anything, unreliable equipment. I cannot overemphasise enough just how critical it is to both my productivity and my mental state that stuff just works. Always.
When it came to replacing the PC, I wanted something that was a turn-key solution. I have absolutely no interest in building PCs anymore and yes, I had a lot of fun doing that for many years but no, I don't have the time nor inclination any more. I'm happy mucking around with things like IoT all day long but that's not critical to me getting work done! I also wanted something rock solid and super reliable so I did what I've done with every laptop purchase I've made in living memory - I decided on a Lenovo (I'd had ThinkPads for years when they were under IBM as well). As I wrote many years ago, I'm part of Lenovo's Insiders program and they do often send me machines, but I'm also confident enough in their product that I'll spend my own hard-earned cash with them. For full transparency, I do get a discount when purchasing from them so that helped the ROI (ok, I probably ended up spending the same amount, I just upped the spec!) but I want to be clear that my money would go in their direction regardless.
In terms of my requirements, I had a bunch of criteria I never actually wrote down until now, but better late than never:
I do a lot of data processing in SQL Server which can be really memory intensive (no, the cloud is not always the solution to this, I've got a draft blog post on that)
I multi-task like a demon: SSMS plus Visual Studio plus Adobe Premiere and Photoshop plus 100 open Chrome tabs plus email plus background music (there's a separate discussion around why I can't just focus on one damn thing at a time...)
I wanted a tower large enough to fit up to 4 additional 3.5" HDDs (I often split processing across disks)
The GPU had to be capable of driving 3 high DPI displays and have an appropriate number of ports to do it
It needed to be quiet to get the most out of that nice new mic
But which Lenovo? Back to paradox of choice again here because there's a heap of options. What started to become apparent though was that a ThinkStation P Series was the right path forward, but then you've got a heap of options still even then:
In simple terms, my logic went like this: how much can I justify spending and what's the most machine I can get for that price. A quick side note: a bunch of people said "get an AMD Threadripper" but at the time, everything in the Lenovo series was Intel-based. They've since launched the ThinkStation P620 although it's still not on the Aussie catalogue.
tl;dr - I went with the P720 as I could spec it with dual Xeons and I didn't need a case as large as the P920. I configured it as follows:
4x 32GB RDIMM DDR4-2933 ECC (there's another whole lengthy discussion on how RAM should be split across DIMMs and the channels they sit in)
1x 1TB SSD M.2 PCIe NVMe Opal (runs the host OS and I put in additional existing drives for other purposes)
1x NVIDIA Quadro RTX 4000 8GB (comes with 3 Display Port sockets)
A 900W power supply (never know how many additional drives I'll end up running off it)
It's a beast, plus there's a 3-year onsite warranty and again, for my purposes, that's actually really important stuff these days. Here's what's inside:
It's all pretty compact and TBH, the two additional drives I've already slipped in were a tight fit. But it's compact in a very organised way and I'll have no problems dropping more in later on. The only criticism I have of the machine (which I'll raise here while the pic is in context), is that the cover is a nightmare to get back on. You can see how many points the cover needs to attach to (4 on the top, 4 on the right, more on the bottom and left) then it locks into place (literally with a key) via a plastic latch that attaches to the metal protrusion mid-right of screen. I've fed this back to Lenovo, they were obviously aware of it already and advised it'll likely change in the future based on customer feedback and that the best way of getting it closed is to lay the machine flat which worked for me. Ultimately a small complaint in the scheme of things, but a complaint nonetheless.
Moving on, it's quiet, really quiet. I don't actually know just how quiet because it can't hear it over the UPS! More on that later but the main thing for now is that it's very close to silent and it certainly doesn't make any noise that's picked up by the mic.
Performance wise, what can I say? It's fast? The thing about a quick PC is that for the most part, it just makes things you'd normally do more efficient as opposed to being a big obvious change to the way you work. It's also still early days (it arrived last week) so maybe I'll discover more about it later on. For now though, it does everything I expected.
Ok, so this is the big upgrade and only a picture (or video) of it all even begins to do it justice:
Seriously, I love this setup! Strictly speaking, I didn't need to replace the screens I had before, but they were now the oldest pieces of kit in the whole setup. The image earlier on of my 2009 setup had 2 x 24" Samsung screens on them running 1,920 x 1,200 each. I'd bought them cheap second hand off eBay and I added a couple more later on giving me 4 screens in total. It looked like this:
After my mic died last week, the water cooling on the PC has also packed up after ~6 years. Along with other issues, it’s new machine time which I’ll order today. Will also replace monitors after more than a decade and thinking 3 larger higher def units. Suggestions? pic.twitter.com/w71QmRsSZW
Not terrible. The main issue I had is that there was always a bezel smack bang in the middle of everything. I ended up with the webcam on the second monitor which was angled slightly inwards which made for an odd angle on the camera IMHO. I tossed around different ideas and again, went through that whole paradox of choice thing. The main consideration was whether I should get 3 x 4K screens or go with an ultrawide which tends to be a lower pixel density. Plus, there's the whole discussion around flat versus curved and it seems to be rather polarising. I turned to Twitter:
Alrighty, I’m getting heaps of conflicting responses so it’s poll time: if you’ve used an ultra wide curved monitor in the past, is it a good thing? Purpose is mixed use and includes coding, reading, video editing and posting photos of Australian beaches. Should I go big curved?
That pretty much sealed it for me, despite there being some comments in there from people hating curved screens. Obviously, I decided to give it a go anyway but needed to find the right model now. The penny dropped early on that I could put an ultrawide in the middle and then two 27" screens either side which, I posited, would look awesome. But to maximise the awesome, I needed both screen types to come from the same manufacturer and look like they're from the same family. I settled on Samsung, namely the CRG90 for the ultrawide and 2 x CJG54s for either side. Sat next to each other, you can see why I choose these two:
The ratings of each were positive too, of course, but the main thing was to get a total of 3 screens with the same number of vertical pixels (they're all 1,440 high) spread across the same vertical space (the display sizes are identical, although the bezel on the ultrawide is slightly thicker) and sitting on similar stands (which I later discarded anyway). Oh, and just for reference, that ultrawide is effectively two 27" screens side by side so in real estate terms my rig has gone from 4 x 24" screens to 4 x 27" ones.
So what's it like to use? Fucking amazing 😎 Seriously, I really didn't know if it'd gel with me or not, but I absolutely love this setup. There's obviously a heap of space but what's most important is how it's used. I'm typing this now with the browser dead centre and occupying a third of the screen width. This is my focus right now and being in the middle does wonders for that. Either side on the ultrawide I have a bunch of other reference websites. Over on the left screen is my email (out of direct eyesight because I don't want to deal with it right now) and over on the right is, well, even more browser windows (I seriously need to close some tabs...)
One of the best things I did to manage screen space was to install PowerToys and use FancyZones. This allows you to divide any screen up into logical sections then drag windows into them by holding down the shift key as you do it. I played with a bunch of configs but always come back to dividing the ultrawide into horizontal thirds:
I'm still finding the best way of using these screens but here's a totally unexpected use case: when I was investigating the Wattpad data breach, I wanted to see each row of user data line by line. Word wrap sucks for this as it's hard to eyeball whether, for example, any of the DOB columns were completed. Stretch it out the full 5,120 pixels though and it looks like this:
You can't see the details of the rows, but it doesn't matter, main thing is it filled out almost the entire width of the screen at normal font size and made my job a hell of a lot easier. I can't begin to explain just how useful that has been! Stuff like this saves time and yes, I know it's cliché, but time is money (or time I could be spending with my kids).
Problems? There are a few things that require adjustment, for example if I'm watching a YouTube video and want to maximise it, well, here it is:
Consequently, I find myself dragging video over to a side screen.
Then there's the screen sharing argument:
Looks like a screen-sharing nightmare. I don’t buy into this kind of setup unless you are gaming or a stock trader 🤷♂️ https://t.co/4fBzCUxFvu
There are 2 easy fixes to this and one of them is staring me right in the face from either side of the ultrawide - just share content from a "standard" aspect ratio screen. The other fix is the very high-tech approach of simply changing your resolution:
This puts a 16:9 screen smack bang in the middle of the ultrawide. (Incidentally, I'm yet to play a game or trade a stock on this setup 🤷♂️)
Finding the mouse can be a challenge, but usually giving it a little jiggle will surface it from somewhere. Either that or I'm trying the "hit the CTRL key" option which is easily configurable. It's a minor sacrifice to make for all that screen real estate.
A bigger problem I have is when opening a window or a dialogue box and it just appears somewhere totally unexpected. It's not an issue most of the time but some apps seem super flaky that way and the problem can be exacerbated by having loads of other windows open on the screen at the same time. Again, a minor sacrifice but noteworthy all the same.
The final thing I'll add on this is that the ultrawide is more than the sum of its parts. I'll take one ultrawide over 2 x 27" screens any day of the week. Not having a bezel down the middle is amazing and there's something about being able to dock windows you're working on right next to each other which is just really hard to put my finger on. It's like this seamless experience where everything just flows whilst before, those bezels were kinda psychological barriers keeping things apart. I can't see me ever going back, it just rocks 😎
I hadn't originally planned to go down this path, but it's made a massive different to the cleanliness of my desk. Seriously, I can't overstate how nice it feels to have more clean, free space around me.
The trick with monitor arms is that they aren't all made equal, far from it. I saw a great 5-minute video whilst researching arms that explains (and shows) just where the difference is between a $25 arm and one that costs 12 times that:
There's a section in there about whether the monitor will support the display and that was a key decision point. Because of the way the arms are designed to support the load, I was almost certainly going to need a different arm for the ultrawide monitor than what I was for the two smaller units. After much research, I ended up with precisely the arm featured in that video - the Ergotron. More specifically, the Ergotron HX for the ultrawide:
And they look... spectacular (I'll come back to the lighting a little later on):
Because they occupy such a small amount of space on the desk, I've been able to optimise the area with the things I actually need as opposed to consuming a huge amount of real estate with chunky monitor stands. You don't realise quite how much space this saves until you look at the original stands:
I never even unwrapped the two smaller ones (the ultrawide screen arrived before the desk mount did) and I don't expect I ever will.
One more thing - it was a bit hard finding the Ergotron arms in stock and I had to do a lot of trawling around various stores before locating them. I've heard the same thing from other people too so do keep that in mind if you're wondering why you can't easily locate them.
So this was another big one in terms of environment change and it wasn't something I was originally planning, until I asked this question:
Alright folks, just 2 weeks until the end of the financial year here: what (legitimate!) business purchases are a "must have"? What would you spend your dollars on in order to make you more productive?
Of course, you need to be a bit sensible here: I can easily reduce my tax by blowing a heap of money on legitimate tax-deductible expenses, the trick is to ensure the dollars are spent on something that gives you a return. I spend a heap of time in front of the camera either doing my weekly update vids, having conference calls with people or any manner of other things that are now online. Plus, let's face it, we're heading into a future where much more time will be spent in front of a camera at our desks instead of in front of movies on long haul flights.
I'd previously been using a Logitech Brio because firstly, it's pretty high spec as far as webcams go and secondly, it integrates with Windows Hello in order to unlock your machine based on facial recognition (it also has an IR sensor which is required for this). Thing is, the quality is ultimately just "ok" (refer back up to weekly update 198 where I demo'd the new mic) and I was broadcasting my head at way under optimal settings.
The solution is to get a proper camera, which takes us back to the paradox of choice again. I must have blown days working out which way to go here, not just in terms of camera but lens as well then how to get the signal out of the thing into the PC and have it behave like a webcam. It's not like you can just take, say, a Canon DSLR and plug it straight into your computer. (Before people start correcting me, the recently launched EOS Webcam Utility has very mixed reviews and especially still being in beta, I just didn't want to rely on it for a big part of my livelihood.)
After much procrastination, I ultimately settled on a Sony a6400 with a 16mm Sigma f/1.4 lens. This was consistently the mix that rose to the top as I browsed around the web, asked on social and spoke to camera stores. There are many pros for this setup: the camera body isn't too large and can easily mount on a tripod, it can do HDMI out without overlaying it with other info, the lens works great in low light, creates a great bokeh effect on the background and focuses really quickly when the depth of field changes. This combo does excellent face tracking too and always keeps you in focus.
When I first got the setup, I did a weekly update vid which I recorded in 4K directly onto the camera. Here's how it turned out:
So that's the camera and lens, the next trick is to keep the camera powered the whole time and be able to treat it like a webcam. The first bit is easy and is done with a "dummy battery" which is essentially a battery-shaped unit with a power cable hanging out the bottom. Here's the one I picked up on Amazon:
Behaving like a webcam essentially boils down to running HDMI out of the camera into a USB dongle that the PC then recognises as a traditional webcam. You can go super cheap and basic such as this A$33 unit, but I just didn't have the confidence in this for something which was going to be such a critical part of my build. The "gold standard" which consistently came up in discussions is the Elgato Cam Link 4K:
There are only two problems: you're looking at 10x the price and you can't buy them anywhere at the moment. Actually, you can't buy them anywhere at retail right now (seems like everyone has the same idea about staying home and streaming content), but there are a few on eBay. They were all selling at higher than retail price too, so it stung a bit to do this, but it's doing it right. End result? Here's last week's vid live-streamed to YouTube in 1080P:
4K is massive overkill for live-streaming, plus I was also seeing a delay between what was appearing on the camera's screen and on the YouTube stream (the one shown directly on my PC, not the one the masses were seeing) which I assume is related to the higher bitrate requirement.
Lastly, there's the question of how it's all mounted. At present, it's sitting on a Joby EverPod Lite tripod:
That's sturdy enough to hold camera yet "lite" enough to easily move around so I just place it in the middle of my desk when I use it, but ideally I want the camera permanently mounted from behind the ultrawide. For that I have an Elgato multi mount "modular rigging system" on the way:
That'll give heaps of options and my hope is that I can just mount it behind the centre speaker then swivel the camera out when needed so it'll be near invisible when not in use. We'll see, I'm not convinced yet, but it was the best option I could come up with for the moment short of physically mounting it to the wall on an adjustable arm (and I'm not ruling that out either).
With no more Logitech Brio, I needed another means of biometric auth. My Microsoft password is definitely not fun to type in, plus I want to be able to auth with other people looking over my shoulder. It was time for a fingerprint reader and the most consistently positively reviewed one I could find is the Verifi P5100:
It's nicely built with a solid aluminium body and sits discretely off to my right, away from the main desk. It's a tad slow to read (the built-in ones on my Lenovo laptops are faster), but not as slow as the Brio was and more reliable too. I'm using this all the time not just to unlock my PC but to auth to 1Password as well. Highly recommended!
So how about those lights earlier on?! 😊 Let's try the same view with a different lighting effect:
Obviously, I spend a lot of time in front of the camera and this is the setup I've had running for years now, a couple of big Fovitech panel lights bounced off the white wall to soften the image a bit. It works great for when I'm on camera, but it's stark. There are halogen bulbs in the roof but they're also a stark light and a dodgy dimmer keeps causing them to flicker lately too. So I added mood lighting.
The lights in the colourful pic are a couple of Hue Go units on the desk and a Hue Lightstrip underneath it. They're all completely customisable in terms of colour and intensity so I don't expect to maintain such "gamer" colours, but it does make for a pretty cool effect. Truth be told, one of the reasons I put the Hue units in was because they can also act as a Zigbee repeater. Without going down yet another massive rabbit hole, a bunch of the IoT sensors I have around the house (for example, for tracking the temp in each room) didn't have sufficient signal strength to talk to the ConBee II sniffer on the bottom floor of the house inside the server cabinet inside the garage and the Hues act as repeaters. It looks like this:
That's probably already more than you wanted to know, point is there's actually a functional purpose behind those lights as well. Also, by being IoT enabled I can do a heap of other things with them using Home Assistant (another massive rabbit hole). For example, I have a few little motion sensors I've been using to setup automations in the kids' rooms (they walk in and lights come on, walk out and they go off). I'll use one of those to turn these lights on when I enter the office and flick them off if there hasn't been any movement for a while.
When I asked about legitimate business expenses for the end of the fin year, another great suggestion was the Elgato Stream Deck:
This is plugged into the PC via USB and is effectively 15 customisable hot keys with LCD displays on each you can tailor. The value proposition is that they're programmable and make it easy to perform pre-configured tasks with a single push. For example, a lot of streamers use them to automate tweets, voice effects or other repeatable actions. I wasn't entirely sure what I was going to use it for when I bought it, but I'm gradually finding some good cases.
One of the best uses I've found so far is to toggle the aforementioned lighting in my office. There's a button for the coloured "mood lighting" and another button that turns all that off and toggles on the panel lights (they're sitting on an IoT power switch). It means that those two very distinct ways of working are just a click away which is especially handy when someone suddenly wants to go from audio to video and I'm not prepared. And just in case you're trying to work out how to do that, it's connected to Home Assistant with web hooks triggers defined on both automations then I used the Home Assistant Webhooks for Elgato Stream Deck plugin to wire buttons up so a single press toggles modes. It's kinda awesome 😎
(Sidenote: one other fun use case is I've pulled the audio from sadviolin.com and wired it to a button in preparation for the next time the kids come into my office and tell me how hard their day at school was 🤣)
Mouse and Keyboard
We're getting to the pointy end of things now, but it'd be remiss of me not to talk about the very devices my fingers spend all their time on. Many years ago, (still in my 20s), I started getting RSI from all the time I was spending on mouse and keyboard. After physio and advice on ergonomics, I moved to Microsoft devices designed, in simple terms, to make your hands and arms last longer. Ever since then I've stuck with various iterations of their input devices as you would have seen in the original ultimate office blog post.
These replaced ageing variants of the same things but wow, what a difference! The obvious one when ordering is that they're both Bluetooth so no dongle like I'd had on the last one hence more free USB slots in the PC. What wasn't obvious when I ordered them is just how nice they feel. I mean really nice and that's a combination of the felt on the palm rest of the keyboard, the smooth feel of the keys themselves and likewise, the surface textures of the mouse. But for some reason it's the smooth scrolling of the mouse wheel I'm loving most; it doesn't click on scroll like my last one and I'm really liking the lack of haptic feedback for that purpose. I know I'm waxing lyrical about otherwise pretty boring devices, but I just love these two units.
To top it all off, I found a wide felt mouse mat on eBay of unknown brand. It's probably the cheapest thing I bought in this whole setup, but I just really like how it frames the whole workspace.
Last thing - power. I've had UPS units before with the last one being an APC job. It died. Early. As with many other decisions in this build, I turned to Twitter:
Alright folks, UPS time. I need a rack-mounted unit to power the @ubnt Dream Machine Pro and Gen 2 switch along with a @synology DS1512+ NAS and modem. Mostly to protect against surges, mains power cycles and short term outages. Suggestions?
And also, as with many other decisions, there were way too many choices. However, Eaton regularly floated to the top and I ended up with an Eaton 5P 850VA / 600W Line Interactive Tower UPS:
The criteria for choosing this basically boiled down to figuring out how much power essential units would draw, how long I wanted them to remain powered for in an outage and then what form factor to choose. In the end, this little unit is driving the PC (obviously), the ultrawide (I should switch this to one of the 27" screens that'll draw less power and provide more battery life), the switch in my office (which provides PoE to the access point here) and my old Logitech Z-5500 speaker setup. Why the speakers? Because running the power through the UPS has completely solved this problem:
Alrighty, my faithful @Logitech Z-5500 buzzes in a (very) noticeable fashion when powered on. Away from all other cables and interference, unplugged from speakers and control unit, just hums away as soon as powered up. Ideas? pic.twitter.com/q8tifugfDT
There must have been some line noise somewhere and this has now fixed it right up. Driving all that, the present load on the UPS would give me 14 minutes of life from the battery (according to the inbuilt display) and that's enough to either get the power back on if something trips it or failing that, save any important work and shut down gracefully. I also ordered a rack mounted Eaton unit for the server cabinet, but that led to other (now resolved) problems I'll write about another time.
The only complaint I have about this unit right now is the fan noise. It's not enough to be picked up by the mic, but it's enough for me to hear something and I really like being able to have just a super quiet environment. I foresee me building a soundproofed enclosure for this one...
I love this setup and as of right now, there's nothing I'd do differently. There's still tweaking to be done and that's everything from the Stream Deck keys to the camera mount to the lighting, but I'm sure that's something that'll evolve over time. For a set of gear that's literally the primary source of my entire professional livelihood, this config absolutely nails it and I can't recommend it all highly enough.
I love beer. This comes as no surprise to regular followers, nor should it come as a surprise that I maintain an Untappd account, logging my beer experiences as I (used to 😢) travel around the world partaking in local beverages. When I received an email from someone over that way who happened to be a happy Have I Been Pwned (HIBP) user and wanted some cyber-assistance, I was intrigued. You'll never believe what happened next...
The tl;dr is that someone with a BeerAdvocate account was convinced the service had been pwned as they'd seen evidence of an email address and password they'd used on the service being abused. They reached out to my guy (we'll call him that for the sake of brevity) who then reached out to me. The relevance to Untappd is that they both share the same parent company (Next Glass) which picked up BeerAdvocate earlier this year and inevitably, they also now share some of their human resources.
In february 2020, I received an email from Netflix that I had signed up for an account. This was to an email address that is completely unique to beeradvocate (as is the custom I do with many sites I sign up for). Someone had registered a new Netflix account with my email / password associated with my BeerAdvocate account. This email address & password combination has existed only in two places: my memory and beeradvocate's database. Not even a password manager.
My going in position when contacted was that this would be yet another case of someone unfairly misattributing a breach to an organisation based purely on what they believe to be a unique email address or password being used in a way they didn't expect. I see this all the time and I literally have a blog post in progress titled "Has a Site Been Breached Because I Received an Email to an Address Unique to Them?" It details many different reasons for this behaviour that are entirely unrelated to a breach and in my experience, there is almost always a non-breach explanation. But not this time.
Plugging the email address in question into HIBP resulted in only a single hit:
Unverified breaches are incidents where the data is legitimate (for example, people's real email addresses and passwords), but I haven't been able to confirm the legitimacy of the source. Per the description in the breach above, that incident definitely had data that could be traced back to both Coupon Mom and Armor Games, but what else might be in there? I pulled out the original breach and searched for "beeradvocate". 816 rows came back:
Well that's... damning. You simply don't have that many matches without there being a very high likelihood BeerAdvocate had suffered a data breach. For every one instance of an email address or password with the string "beeradvocate" in it there'll be another 100 instances that still came from their service but didn't use a customised email alias or (let's face it) very poorly chosen password. On the balance of evidence, they had indeed been breached and their data rolled in with at least the two other organisations into what was now effectively a credential stuffing list.
After a thorough investigation from an independent third party cyber security firm, it was confirmed that BeerAdvocate user login credentials (email address, BeerAdvocate forum password) were lost and aggregated along with breaches of other websites into a breach dataset that became known as CouponMom 2014.
I'd argue that they're not lost, instead there's actually a lot of backups of them! They dated the breach back to "seven or eight years ago" and stated that "a since-retired password hashing method allowed some passwords to be derived". They don't state which algorithm was used, but it's a safe bet it was MD5 or SHA-1 which was already pretty fundamentally flawed by that time. I personally would have approached a number of things around this incident differently, but Next Glass still deserves some kudos for taking the concerns of the individual who raised this seriously and seeing it through to its conclusion, especially given they inherited this breach by virtue of the BeerAdvocate acquisition.
Just one more thing - I've often been asked why I don't discard the source data of a breach once processed and email addresses loaded into HIBP. Putting aside the fact that discarding it doesn't actually make it go away (a quick search found this data still being extensively traded), historical breaches can be enormously useful in establishing the origin of subsequent breaches. This incident exemplifies that and without ready access to this data I don't know that BeerAdvocate would have established the breach, notified their customers and given them the opportunity to go and change that same one password they use across all their other accounts...
There was much applause which I countered with "is it a joyous moment, because it's kinda sad as well?" But what's even sadder than 1B breached records is 10B breached records:
New data breach now loading into @haveibeenpwned that'll push it *well* over 10,000,000,000 records. Wow. Insane, never thought I'd be here doing this with those numbers. It's been a fun little project 🙂
I fired that tweet off whilst loading the Wattpad breach without giving it much thought, but based on the likes it received, it seems to have resonated. On reflection, what really struck a chord with people is that despite the raw numbers, HIBP remains precisely what I concluded that tweet with - a fun little project. Something I enjoy running purely for the pleasure of creating a service that other people find useful. And that got me thinking a whole lot more about the purpose of HIBP.
Earlier this year, I wrote about how the M&A (merger and acquisition) process that consumed most of my 2019 and a substantial part of my sanity ultimately resulted in a "no-sale". Reaching the other side of that process - regardless of the outcome - was an enormous relief. The highly emotional, time and money sapping, single most stressful thing I had ever done in my professional life was done. I wrote about it publicly as soon as I could then got back to running the service as usual, not giving it much further thought until just now at this 10B milestone. Just as I said in that tweet, I created HIBP for fun and it was always intended to be a community-first initiative. As I opened with in this blog post, it was just a pet project:
A project, activity or goal pursued as a personal favourite, rather than because it is generally accepted as necessary or important.
By way of the M&A process, I was forced to redefine that, at least on paper. I found myself thinking less about SQLi and more about EBITDA; that's just not HIBP and it's not me either. The whole prospect of "value" was becoming this really wacky world where people were trying to put a price on something I did for the greater good and turn it into a tradable commodity. Zack Whittaker published a fantastic piece earlier this month on How Have I Been Pwned became the keeper of the internet’s biggest data breaches where he wrote the following:
Buyers were only interested in a deal that would tie Hunt to their brand for years, buying the exclusivity to his own recognition and future work — that’s where the value in Have I Been Pwned is.
It wasn't even HIBP that was the tradable commodity, it was me and that just didn't feel right. With the benefit of many months of clarity since that process, I can say one thing for sure: that's never going to happen - ever!
I love the relationships I have with so many of the various organisations I've worked with over the years, but I love the ability to choose them of my own free volition even more. I love that I can choose to work with these organisations be it related to HIBP or not and equally, I love that I can choose to sit on the beach and do nothing. I also love that there's so much community support for precisely this approach, especially around not putting a price on HIBP and shipping it off to a bidder somewhere on the other side of the world. If there was ever any doubt about the support for keeping HIBP as first and foremost, a community project, a quick read through the responses to my tweet about it remaining independent will dispel them. Now more than ever, suggestions such as open sourcing the whole thing make a lot of sense and that's something I'm seriously considering, along with working alongside non-profits who can prioritise the service, not the dollars.
And that, to me, is the essence of HIBP: community first. The ability for people to freely learn about their exposure in data breaches without concern about my motives or influences is the heart of this service. In turn, it's the support from the community that's made it possible in the first place whether that be all the expertise people have lent me over the years, the services organisations like Cloudflare have provided and indeed all the individuals who's sourced and contributed the data. You can't put a price on that and perhaps, in the end, that's what the failed M&A process proved. I can't think of a better outcome for my fun little project 😊
I made it to 200! And look at that picture quality too 😎 I'm streaming in 1080p rather than 4K and that's absolutely fine for content like this. I've finally gotten on top of the camera setup and the Elgato HDMI dongle to allow the camera to be seen as a webcam over HDMI. I really want to write this up in detail for next week's update because with the new PC as well, I'm super happy with how this all works together. I'll try and put aside a day early next week to get on top of that one but for now, here's this week's:
Wow! Loving that 4K camera 😎 Or perhaps more specifically, just loving that camera and lens and I reckon it'll still be awesome in 1080p. But this week, I decided to go all out in super hi-def just to see how it looked. The captured video was 13.1GB but rendered down at 2.3GB out of Premiere so it's obviously applied some compression, but still looks amazing IMHO. Next week I'll do a full run through of the new setup which I'll finally be able to do because just as I finished recording today, this turned up:
Well, no surprises here: this week's update is dominated by Thursday's blog post about sustaining performance under extreme stress. The feedback on that post has been absolutely phenomenal; tweets, comments, DMs, emails, phone calls, all enormously supportive. Many of them also shared people's own personal struggles, ones which I think we all know are out there but it's a very different thing to actually hear it from someone personally. Thank you everyone who chimed in on this discussion and offered their support and kind words, it's genuinely appreciated and it's made a big difference to how I feel about the last 18 months.
I started writing this blog post alone in a hotel room in Budapest last September. It was at the absolute zenith of stress; a time when I had never been under as much pressure as I was right at that moment. Project Svalbard (the sale of HIBP which ultimately turned out to be a no-sale) was a huge part of that and it was all happening whilst still being solely responsible for running the project. That much was very broadly known publicly, but what I haven't spoken about until now is that earlier that year, my wife and I had decided to separate and later divorce. As part of attempting to rebuild my life, I was also in the midst of buying another house, a stressful process at the best of time let alone under these circumstances whilst on the other side of the world. It was extreme stress the likes I'd never dealt with before at a time when the demands on me were at an all-time high, so I started writing this blog post, adding to it at the worst of times. Here's how I sustained my performance whilst under extreme stress:
I Leaned on Friends More Than I Ever Had Before
I realised something very profound last year; I've very rarely discussed my emotional state with friends. Maybe that's "just what blokes do" (or don't do), but it certainly wasn't a conscious decision on my behalf. It wasn't until the stress really started mounting early last year that I actually made a conscious effort to do this. Putting it in words now seems almost stupidly obvious, but there's a lot of evidence around the benefits of friendship on mental health:
It can be hard to talk to family members about mental health. That’s why it’s important to have healthy friendships to turn to in times of need. Our friends can be that ear to talk to, shoulder to lean on and nonjudgmental perspective that we need. They can also help increase our sense of belonging, improve our self-confidence and help reduce stress and anxiety.
Last year and early this year, it meant spending a bunch of time with friends in person during my travels. Since Feb this year as travel has become a thing of the past, it's meant talking to friends in different parts of the world every couple of days. Often those discussions have directly focused on the stresses in life but equally often, they've been an opportunity to bond around less contentious common interests; cars, tech, family. The quote above about helping to increase a sense of belonging really nails it.
The thing that perhaps surprised me most about those discussions with friends was how much their own stories resonated with mine. I mean that across all the fronts I was feeling the stress on too; whilst in San Francisco in particular, I spent a bunch of time with people I knew well who'd been through similar business processes and as for the things stressing me in my personal life, it felt like every second person I confided in had a similar story. Finding common ground with friends was always a huge relief; I wasn't alone in what I was going through.
I Did My Utmost to Not Make Decisions Based on Emotions
Emotions have been high during this period, both professionally and personally. More than anything, it was the unpredictability of emotions that got me; I could be cruising along thinking everything was on track then wammo! An email, a text message or a phone call would suddenly throw everything back into turmoil. I'd be upset. Angry. Vengeful. But none of these feelings would help me make rational decisions.
Frequently, I'd simply sit on an email for a day. I'd sit on my own emails for a day, granting time to reflect on whether my words represented the best path forward or merely reflected my emotional state at the time. A perfect example is that the house purchase fell through due to the vendor not being agreeable to the terms I set forth. I received their reply and was initially upset. I sat on the email, went and did a conference talk, drank some beer, had a sleep and responded the next day, cancelling the deal. It hurt to do that because I really wanted the house, but I also knew that "want" wasn't enough, it had to actually make sense and without agreeing to my terms, it simply didn't.
I haven't always gotten this right and there hasn't always been the luxury of time between emotion and response, but as a strategy to keep peace and maintain sanity, it's proven invaluable time and time again. I can't think of a time where I slept on a response and didn't tone it down a bit.
I Stayed Focused on The Bigger Picture (and the Small Steps That Would Take Me Towards it)
I was always looking a year or more ahead and I had a very clear picture in my mind of how I wanted my life to look like in the future. Stress has a way of clouding judgement and causing you to make irrational decisions, many of which might feel right at the time, but don't ultimately further your life goals. I had a vision of what my future would look like (and obviously given the HIBP no-sale, reality hasn't always aligned with the vision), and everything that was happening as I wrote this blog post had to support that objective. But there were also massive changes in my life that had to be dealt with here and now, and there was only one way to do it:
Consider what was required to achieve the big picture goals I had; everything from literally hundreds of meetings, thousands of emails, endless proposals, terms sheets, negotiations - and that was just on the HIBP front. Throw in the stress, emotion and frankly, some pretty dark moments on the relationship side of things and consider how totally overwhelming it can all feel.
I tackled it by focusing on the very next thing I needed to do to; the single, attainable thing I could do to move me towards a goal. Complete some financial documents. Schedule a meeting. Agree on some key deliverables. So long as the activity was an enabler of that big picture it didn't matter that it was a little thing, it was progress.
I can think of many occasions across all the various things that put me under stress this last year and a half where I literally concluded "fuck it - it just doesn't matter enough". They were things that by any reasonable measure I had every right to be upset about, but equally they were things that had I gotten upset about them, they'd derail me from focusing on that bigger picture.
Legal jargon in contracts is a prime example. I recall one occasion where lawyers on my side of the HIBP deal were arguing with lawyers on the other side about whether or not I was a "sophisticated investor". I needed to be in order to receive the proposed equity component and unless we agreed that I was, the exact words I heard were "the deal's off". It was an obnoxious comment about a ridiculous premise, but ultimately, we concluded that the real world impact of the clause was likely negligible and further arguing about it really didn't serve my own purposes.
I Moved on Quickly from Setbacks
There were so many outcomes along the way that frankly, felt devastating. Incidents and events that left me fuming, emotional and sometimes, pretty inconsolable. It was so easy for these things to eat me up and consume me, taking my focus away from that big picture and keeping me from moving forward towards that bigger goal.
I found I kept going through the same cycle after a setback and it tracked pretty closely to the whole Kübler-Ross model of 5 stages of grief. I'd very quickly move through denial and anger, blast through bargaining and depression and get to acceptance. I tried hard to bring myself to that last stage and I remember thinking so many times on the way there "this feels much worse now than it will tomorrow or the next day".
In thinking of an example to illustrate this, the following tweet and exert from the "no sale" blog post came immediately to mind:
Project Svalbard was the initiative to find a new home for @haveibeenpwned. After 11 months, the project has now run its course; HIBP will remain independent. Here's the full story: https://t.co/euM50h21Ge
According to the lock screen, I took the photo below at 04:49 on the 24th of July last year. I was in yet another bland, nondescript hotel room, drinking bad coffee in an attempt to stave off the jet lag. I'd arrived in San Francisco a few days earlier after barely making my connection in Helsinki, literally running through the airport. My bag hadn't made it. I was tired, alone, emotional and if I'm honest, at an all-time low.
I felt like shit at that moment, but it was temporary and I had just enough sanity left to know that the feeling would pass. Just. But it always did pass and there'd be something else of a much more positive nature happen the very next day.
I Always Thought 3 Steps Ahead
Let me begin by saying this: I didn't always get this right (far from it) and on multiple occasions I got blindsided by things I never saw coming (the circumstances under which HIBP ultimately didn't sell is a perfect example). But the basic premise is that before expressing my position on something, I'd consider the range of possible responses I'd receive. Let's say there were 3 of them; for each of those 3 possible responses I'd not only consider how I'd respond to each, but how each of my responses would then be received. Same again for how I'd respond to each of those and in my mind, I was drawing out a mental image of 3^3 different possible outcomes - which one did I want? It was an exercise that enabled me to look much further down the road and consider whether it aligned to an earlier point in this blog post - my big picture.
This requires time, practice and patience and as I said in the opening, I didn't always get this right. You can't always be aware of all the factors influencing third parties nor can you be aware of all the cards they hold, but without doubt, this way of approaching any negotiation is enormously valuable. It also forced me to empathise; how will other parties feel? What's the most natural reaction they'll then have?
In my mind, this is akin to a "choose your own adventure" book; at each crossroad there are different ways you can go. Each of those then has their own crossroad as do those ones too. Before making a decision at that first intersection, I want to know what the next 3 will look like.
I Drank Beer
Treat this less as a suggestion to consume alcohol and more as a representation of taking time out for yourself. For me, having a beer is something I associate with switching off from the everyday stresses. I very rarely drink alcohol when working (now coffee, that's another story!) and treat beer as an opportunity to "down tools" and relax.
The point is that I made a conscious effort most days to tune out and give my brain a rest. A good mate of mine is convinced meditation is an equal of beer in terms of helping him disengage from daily life and maybe he's right, I just don't have the patience for it (yet). Find your beer, whether it be actual beer or an activity which allows you to do what the process of going and having a cold one does for me.
I Threw Myself into Exercise and Health
From beer to physical wellbeing: I was trying to find a tweet to illustrate the point, and this one nails it:
At this time, I was now well into Project Svalbard, I'd separated from my wife and per the caption, I was preparing to deliver a keynote at Australia's premier security conference. When I first started to really feel the stress, I absolutely threw myself into exercise:
Gav is both my son Ari's and my own tennis coach. I literally said "Gav, book me in every day at the hottest possible time" and when the weekend came, I'd play with Ari as well. The standing commitment each day forced me to get out on the court and focus on something other than life's stresses.
Per the earlier image, I was also getting right into Poké Bowls which meant a lot of raw fish, brown rice and greens like edamame and seaweed. I'd order it on Uber Eats, it'd arrive at my door and IMHO, it's genuinely delicious. Physical health has a profound effect on your ability to perform mentally, particularly when you're under extreme stress. Exercise in particular has very well-documented benefits when it comes to depression, anxiety and stress.
Despite the emotional turmoil of recent times, I'm in great shape physically with a typical week including running, bike riding, tennis and wake boarding. I'm about to pass 3 months of closing all rings on the Apple watch every single day (amazing how much not travelling helps you do that!) and I can really see those benefits showing in the kids too when they share the activities with me.
I Established Stable Routines
In a tumultuous period like this, it's easy for routine to go out the window. Most people have some form of routine which establishes consistency in their life, for example going to work each day. A regular social commitment. A Sunday roast dinner. I spent 243 days travelling last year so consistency was near non-existent.
A saving grace has been my weekly update videos. Every single week, without fail, I've done the video. Sometimes they've been at the worst of times, needing to record and put my face in front of the world after feeling emotional / jet-lagged / broken (and a big shout out to those who commented to that effect!) But what those videos did was give me a small sliver of consistent predictability in life. During each week I'd take notes on content, pull myself together then sit down and record.
Same again for my blogging and drafting this one in particular was a big part of that. For the last 11 years, I've written about most of the things in my life that have been important. Writing transparently about what's going on in my life has become a part of my routine and indeed, a part of my identity. It feel "off brand", for want of a better term, when I don't.
As things have stabilised this year, I've been able to broaden those routines with regular tennis, time with my family and simply walking down to the beach most mornings:
I snapped that pic last week after watching a humpback whale and her calf cruising by, probably just 50m offshore. It was a moment of reflection following a period of great turmoil; it's been both the highest of highs and lowest of lows. But now, being at home and finally having stability it's crystal clear: this is a routine that's going to stick around for the long term.
This was a heartfelt blog post about some momentous events in my life. By all means, please comment, share your experiences and ask questions but avoid topics related to my relationship. As much as I'm open about the emotions I went through and how I dealt with them, details of a personal nature are something that will remain that way. Thank you.
I'm literally surrounded by broken pieces of half finished repairs. My office is usually a pretty organised place so it's kinda frustrating, but then I'm replacing equipment that's seen up to a decade or more of solid use so that's not a bad run. Amidst all that, I've well and truly gone down the IoT rabbit hole with all sorts of bits now connected through Home Assistant (just understanding the basics of this is actually one of those draft blog posts I mentioned). All that, the usual data breach stuff and more in this week's update.
All my things are breaking 😭 Mic broke, PC broke, boat shed handle broke, fridges (both of them) broke, fireplace broke, roof broke... and that's just the stuff I could remember in the live stream. But in happier news, listening back to that video now I'm really happy with the audio quality of the new mic and I reckon that once the pop filter is installed the sound will be spot on. Hopefully that'll be in place for next week's update, along with replacements for the other broken things! Let's see what else fails between now and then...
Today, almost one year after the release of version 5, I'm happy to release the 6th version of Pwned Passwords. The data set has increased from 555,278,657 known compromised passwords to a grand total of 572,611,621, up 17,332,964 (just over 3%). As with previous releases, I've made the call to push the data now simply because there were enough new records to justify the overhead in doing so.
Also as with previous releases, version 6 not only introduces a heap of new records but also updates the prevalence count on the existing ones. For example, the old favourite "P@55w0rd" has gone from 2,929 occurrences to 3,069 so still a terrible password, just a little more terrible than what it was before.
As the size of the corpus increases, new passwords tend to be less common than those that were already in there. For example, the password "Your password" now makes an appearance as does "bullet_hole" and "Pssw0r". Further, a whole bunch of passwords that, um, well, I can't really print here also make an appearance, but use your imagination and you'll probably be able to work out a few of those.
In terms of the Pwned Passwords service itself, it continues to see steady growth:
I decided to frame that tweet in precisely the same fashion as I did the one in last year's blog post, but that was "only" 16M requests in the previous 24 hours back then. I can't attribute the growth to any one single source, rather a heap of individual cases just like this one:
I have every intention of keeping Pwned Passwords freely available and not requiring any sort of auth at all because I genuinely want to see growth like this continue. Mind you, I also encourage anyone not keen on using the k-anonymity model to just download the whole set and as with previous years, it's all available as either SHA-1 or NTLM hashes (read the rationale behind the choice of these algorithms if you think they look a little dated). If you want to use the API, an improvement to the implementation since the last release is the padding feature I wrote about a few months ago. Today's release changes nothing on that front; the same amount of padding is still used as there's only a 3% increase in response sizes (we catered for way more than that when choosing the amount of padding to use).
If you're not sure whether or not you're searching against the latest data set, check the "last-modified" response header and make sure it's the 19th of June this year (the day I uploaded the data to Azure):
last-modified: Fri, 19 Jun 2020 00:47:46 GMT
All cache at Cloudflare should have been flushed and any searches from here on in should show a date and time around the one above. Because of that Cloudflare cache, anyone measuring response times of the service might also see a small increase whilst those 16^5 different possible hash ranges populate back out into Cloudflare's edge nodes. And while I'm talking about Cloudflare, I want to recognise their support again in providing the services that make Pwned Passwords not just super fast for everyone hitting the API, but also super cheap for me to run:
So that's version 6. No promises on when there'll be a version 7 because as with all the previous ones, it's entirely predicated on having enough new passwords to justify a new release. I'm still dependent on having them in plain text either due to the way they were stored or by virtue of someone going and cracked a bunch of them. With more and more websites actually doing their password storage well, that's all becoming a rarer circumstance. Which is good 😊
This week's update had a bunch of people drop by and discussion tended to jump around a bit, but frankly it's kinda nice to have some interaction in an era where we're not really doing as much of that any more. The IoT topic got some good engagement as did the fact that we "magically" dropped over a hundred active cases of COVID-19 in Australia today (sounds like the gov just reclassifying what's still considered to be an active case). That puts us at 143 remaining active cases in a country of 25M people which is very much heading in just the right direction. Oh - and I finally managed to buy my lift tickets for the snow too so happy days all round 😎