Creating a PHP response time dashboard

Following my previous post about creating histograms from PHP response times, I made a small dashboard so I could see those histograms across each site.

It was fairly simple to make -- I just looped over the last 50000 lines in the log file, checked if each line contained a keyword ("remoteok") and then added the data to an array for further processing. I also added some statistics about the data: overall maximum time and then minimum, maximum, mean, median for the data without outliers.

The small change I had to make to the file processing was actually getting rid of the library I used to get the last X lines. It worked, but was terribly slow and I needed a faster alternative. In the end, I used "shell_exec" to call some built-in Linux commands. It dropped the load time from 11 seconds to 1 second.

This dashboard has already helped me optimize some code.

Example 👉 I generated the following from @NomadList logs and saw those ~400ms requests. Checked which queries & profiled. Found some legacy code + unoptimized SQL. Fixed and time dropped to 17ms ⚡️ pic.twitter.com/dplv9VzQfZ— Daniel Lockyer (@DanielLockyer) November 26, 2018

I also saw that the majority of queries on Remote OK were taking around 115ms. Looking at the logs, this was for the homepage. I decided to profile that page, found some small optimizations and ended up dropping it to 100ms, as you can see in the image at the top. Small gains but it adds up.

Something I found interesting was the histogram for levels.io, a WordPress blog. Whereas the other two are spread across a range, it seems the WordPress one had a ton at a few milliseconds, and then essentially nothing until ~66ms. After thinking about the setup, I realised why. The small times are the wp-cron.php script and other small things that get called. The blog has WP-Super-Cache installed, which caches the PHP output. This delay is the time it takes for the WP framework to load WP-Super-Cache and the cached contents. Quite cool, but still slow!

I am still working on adding features, or looking at other data to help optimize the web apps I work on. The thing I am trying to focus on here is making data easier to understand, so it guides where you should be focussing your effort.

I can help you do this for your website, so email me at hi@daniellockyer.com!