Tech

Find the largest unstaged files in Git

I was recently adding a bunch of files to a new Git repo, but I couldn't be sure if I was about to commit a 100MB file. The following bash command should print out a sorted list of unstaged files and their size, so you can check if you are

Automated GitHub releases for Rust projects using Actions

I've been using GitHub Actions more and more across the projects I maintain. It's a great offering and integrates well into GitHub. There are a few Rust projects I have where I wanted to automate the publishing of prebuilt binaries to the GitHub releases page. I've seen others do it

Node.js memory flamegraphs

If you read my blog or follow me on Twitter you may have noticed I'm a huge fan of flamegraphs. I've previously written about how I use them to profile PHP [https://daniellockyer.com/php-flame-graphs/] and Node.js [https://daniellockyer.com/profiling-nodejs/] applications, and I help maintain rbspy [https://github.

Avoiding double builds in GitHub Actions

I was recently setting up some CI on GitHub Actions and ran across double builds for pull requests. The official documentation is still a bit lacking and it's a problem because it burns 2x the credits. In the end, I made this diff: -on: [push, pull_request] +on: + pull_request:

If you are still using PHP 5.6 or 7.0, you need to upgrade

Last October I made Upgrade your PHP [https://upgradeyourphp.com/], a site to let people know that PHP 5.6 and PHP 7.0 were about to reach their End of Life, and would no longer be officially supported. In the time since, I've been helping people upgrade their servers

Profiling Node.js applications using flamegraphs

This post is a case study of how I profiled a Node.js application, and how using flamegraphs allowed me to make the boot time of Ghost 20% faster. I remember trying to profile Node.js many years ago and it was much harder than it is now. When researching

Suggested settings for Cloudflare

I always recommend to my clients that they use a CDN on their website. These are my recommendations for people using the Cloudflare Free plan, along with some warnings to watch out for. DNS * Status: Orange Cloud - this will ensure the requests go through Cloudflare. Crypto * SSL: Full (Strict)

DigitalOcean Spaces doesn't support HTTP/2

Earlier today I was looking at a couple of sites to see why they were much slower than I expected. I ran them through GTmetrix and saw the following in the waterfall tab. Straightaway I could see that this is the classic pattern produced by trying to load a bunch

Flamegraphs for Ruby 2.6

I recently added support for Ruby 2.5.4, 2.5.5 and 2.6.0-2.6.3 to rbspy [https://github.com/rbspy/rbspy], the sampling profiler written in Rust. I have replaced the original Flamegraph [https://github.com/brendangregg/FlameGraph] library by Brendan Gregg with Inferno [https://github.

Linode corrupted our server and how I got it back up

A few days ago, the server running Nomad List [https://nomadlist.com] and Remote OK [https://remoteok.io] randomly went down. This is the timeline of what happened and how I fixed it. All times are in  Korean Standard Time. 21:05 Pieter [https://twitter.com/levelsio] starts messaging me

Caching static HTML with Cloudflare

By default, Cloudflare does not cache static HTML. [https://support.cloudflare.com/hc/en-us/articles/200168216--Does-Cloudflare-cache-HTML-content-] It's nice when you're constantly pushing updates because the changes are immediately shown. But it does mean that you still have that short delay whilst Cloudflare gets the latest version of your page. If

Rewriting stackcollapse-xdebug in Rust

A week or so ago, I saw the inferno [https://github.com/jonhoo/inferno] project mentioned on the Rust subreddit. It was a rewrite of the great FlameGraph [https://github.com/brendangregg/FlameGraph] library into Rust. All of the work was being livestreamed by Jon Gjengset [https://twitter.com/Jonhoo/

Red flags in website performance

When a client asks me to take a look at their site, I start off by throwing it into a bunch of services to analyze the performance. One of my favorites is GTmetrix [https://gtmetrix.com] because it has a lot of useful metrics and visualizations. The most useful one

Project: Cron Heatmap

Towards the end of last year I made Cron Heatmap [https://cronheatmap.com], a website to visualize your cron jobs and help you find hotspots. The motivation from the project comes from my work over at Nomad List [https://nomadlist.com] and Remote OK [https://remoteok.io]. There are over

Hiding your Linode server IP with Cloudflare

Over at Nomad List [https://nomadlist.com] and Remote OK [https://remoteok.io], we recently switched our server IP to protect against DDoS attacks. Whilst the main sites had been on Cloudflare for a couple of months, the server IP was easily findable on Google and we had experienced several

Google Analytics has an anti-tracking problem

We've always known Google Analytics would be susceptible to anti-tracking extensions like AdBlock [https://getadblock.com/] and uBlock Origin [https://github.com/gorhill/uBlock]. I never really thought about how big that difference would be. It was only when we started using Cloudflare on Nomad List [https://nomadlist.com] and

Easy VPS deployment for Jekyll

My blog is generated by Jekyll and hosted on a Linode VPS. This is how I set up Git to allow me to instantly deploy updates. My setup has gone through a few iterations over the years, as I changed technologies and tried to make it easier. I used to

Generating response time histograms for PHP

Whilst debugging an issue related to response times on Nomad List [https://nomadlist.com] and Remote OK [https://remoteok.io], I started to think about what kind of data we have available to analyse. We keep access logs for all requests coming through the PHP-FPM engine. I briefly wrote about

How to generate PHP Flamegraphs

In my quest to optimise the PHP sites I work on, I found that the Flame Graph repository contains a script to convert XDebug traces. Flame graphs, created by Brendan Gregg, visualize the most frequently hit code from profiled software. The x-axis represents the profiled stack traces, ordered alphabetically, and

Project: Server Help

A couple of months ago I created Server Help [https://t.me/server_help], a Telegram group to help indie makers with any server issues. It's not a big group but people come to ask questions they have about DNS, web server configurations and hosting services. I started it because

How to find slow PHP scripts

It is important to keep a website fast so users have a good experience. Some pages on your website may be slower than others, possibly due to slow backend code. I use this super simple method to find the slowest loading PHP scripts and fix them. {: .center-image } Remember, you can't

Fuzzing Rust

I spent some time fuzzing various Rust fuzzing tools against Rust libraries. This table is a list of all the issues I found and links to my bug report. DateProjectDescriptionLink2017-03-13ntp [https://github.com/JeffBelgum/ntp]Fix panic caused by a malformed inputPR [https://github.com/JeffBelgum/ntp/pull/1] 2017-03-23npy-rs

IPv6 weirdness

We have a weird rise in incoming IPv6 traffic for 12 hours a day between 1900 and 0700 UTC. It's an almost sudden rise so it doesn't seem to be due to a certain geographic region waking up. According to tcpdump, it is directed at port 443 but nothing out

Server monitoring

This post is about how we monitor Nomad List [https://nomadlist.com], Remote OK [https://remoteok.io] and others. Over the years, Pieter [https://twitter.com/levelsio] and I have built up a set of tools to run and inform us when the sites may be experiencing an issue. Cron

How to fuzz a Rust program

Manish Goregaokar [https://twitter.com/Manishearth] wrote a brilliant blog post [https://manishearth.github.io/blog/2017/03/02/mitigating-underhandedness-fuzzing-your-code/] about cargo-fuzz [https://github.com/rust-fuzz/cargo-fuzz] and its use in fuzzing Rust programs. Since that post, I have pushed code to update the default fuzz target template and to

Passive network topology scanning

Back in April 2017 I was assigned a networking coursework for my Computer Science degree. It required us to choose a topic in computer networking that interested us and conduct a research experiment. You can read the original report here [https://daniellockyer.com/content/images/networking-report.pdf].

How to profile your PHP apps

Part of my job at Nomad List [https://nomadlist.com], Remote OK [https://remoteok.io] et al. is to keep the sites fast and responsive, even from an influx of traffic to Pieter's [https://twitter.com/levelsio] latest app. Launches are usually kept controlled by exporting the pages to static