Difference between revisions of "SitePerformanceMonitoringTools"

OurWork

What (summary)

Instrumentation that provides a history of performance statistics for each part of the page load pipeline.

Why this is important

The responsiveness and performance of the site makes a big difference in how many pages visitors will view, and how often they will come back. A poorly performing site will also wear out our active members causing some of them to leave.

DoneDone

Define good and acceptable times for pieces of the Performance pipe

• Max cold-request to fully rendered time for normal pages (including especially front page)
  • 3 seconds is unacceptable
• Max Load Time for special goodness (Batch Patrol ... etc)

Record a history of how long to

• lookup DNS for www.aboutus.org, images.aboutus.org, ... from different parts of the world
• Setup a port 80 TCP connection with each of the squal boxes from different parts of the world
• Load the frontpage without any client caching
• Retrieve a memcache item from each combination of two squal boxes (one client, one memcached server)
• Load the core css files
• Load the core js files

...

Performance Priorities

1. View normal page
2. View random page
3. Edit click until available
4. Save click until rendered
5. Render invalidated frontpage

Instrumentation Steps

1. End to end on each
2. Deploy instrumentation boxes in various locations
3. Determine and instrument the pieces
   • MediaWiki profiling
   • Raw database queries

Steps to DoneDone

• Articulate the request pipeline
  • Identify each request in the pipeline
  • How to perform / retrieve data for each request
  • Assign acceptable benchmarks to each request

• Aggregate pipeline benchmarks

• Push XML results to central HTTPS server
  • Remote location / benchmark results stored in database

• Analyze pipeline benchmarks
  • Graph performance for each location
  • Detailed graph view on each request
  • XML output for monitoring integration

• Integrate into monitoring
  • Dashboard to identify overall health
  • Notifications via email/paging critical problem arises

Potential Hurdles

• False positives
• Caching