Round 18 of the TechEmpower Framework Benchmarks project is now available!

When we posted the previous round in late 2018, the project had processed about 3,250 pull requests. Today, with Round 18 just concluded, the project is closing in on 4,000 pull requests. We are repeatedly surprised and delighted by the contributions and interest from the community. The project is immensely fun and useful for us and we’re happy it is useful for so many others as well!

Notable for Round 18

Nate Brady tracked interesting changes since the previous round at the GitHub repository for the project. Several of these are clarifications of requirements for test implementations. In summary:

  • Thanks to An Tao (@an-tao), we clarified that the “Date” header in HTTP Responses must be accurate. It is acceptable for it to be recomputed by the platform or framework once per second, and cached as a string or byte buffer for the duration of that second.
  • To keep frameworks from breaking the test environments by consuming too much memory, the toolset now limits the amount of memory provided to the containers used by test implementations.
  • The requirements for the Updates test were clarified to permit a single update. We are still considering whether to classify test implementations by whether they use this tactic.
  • The requirements were clarified to specify caching or memoization of the output from JSON serialization is not permitted.
  • The toolset now more strictly validates that responses are providing the right JSON serialization of responses.
  • Cloud tests in Azure are using Azure’s accelerated networking feature.
  • Postgres has been upgraded to version 11.
  • Nikolay Kim (@fafhrd91) explained the tactics used by Actix to acheive record performance on the Fortunes test.

Other updates

  • Round 18 now includes just over two hundred test implementations (which we call “frameworks” for simplicity).
  • The results web site now includes a dark theme, because dark themes are the new hotness. Select it at the bottom right of the window when viewing results.

Notes

Thank you to all contributors and fans of this project! As always, we really appreciate your continued interest, feedback, and patience!

Round 18 is composed of: