How our editorial process works

How our editorial process works

Console aims to become the place developers go to find the best tools. Each week we select a handful of tools and beta releases to share with our subscribers in the Console weekly newsletter. Here is an overview of how we decide what makes the final edit each week.

Every week the Console newsletter features two tools we think are particularly interesting. We provide our own take on what we like - every tool we feature we think is worth checking out - but nothing is perfect, so we also provide a comment about the limitations, having played around with the tool ourselves.

The newsletter also include a section on new and interesting beta / preview releases. This usually features tools that have had a release in the last few weeks, whereas the "interesting tools" section has no criteria for being "new". We don't feature everything - that's what Beta Console is for - so our readers know that if it appears in the newsletter, it's worth a look.

Discovering new tools

To keep track of new and interesting tools, we monitor hundreds of sources from well known places like Hacker News, GitHub and Product Hunt to analyst newsletters, funding announcements and company blogs.

We use Feedbin to monitor newsletters, Twitter and blogs (hurray for RSS!), but for forums and community spaces we manually check these sources on a regular basis. Thanks to the Console community we are also increasingly receiving recommendations from our subscribers that we add to the list to review.

Our definition of a ‘developer tool’ is wide and malleable. Above all, it must be a product that is of use and value to developers. Typically this means we’ll consider anything in the category of APIs, cloud services, databases, data science, devops and security. Anything where a software developer is the primary user. We consider tools from individual projects, open source, early stage companies, educational institutions and the latest releases from established public corporations.

Snapshot of the Interesting Tools shortlist

Selection Criteria

All these sources feed our "Consideration" list. This is an ever expanding list of tools that sound interesting that we will subsequently test and use. It's been surprising how many tools there are, not only entirely new projects but existing tools with lots of users we've never heard about!

We're adding to the list faster than we can get through and review the tools. Sometimes it takes just 15-30 minutes to try something out, especially if it has a good onboarding process and well written documentation. Other tools require more time to play around and understand the use cases. Limitations often only show up after we try certain features, examine the docs, check through open bugs and feature requests.

Of course many drawbacks only appear after a period of large-scale use in production, but we aim to offer a balanced view.  The email is designed to be concise, so the review has a soft limit of 300 characters each for what we like and what we don't like. 600 characters total is quite a difficult constraint to work with!

At both stages of consideration - getting on the list and then the review itself - we apply our selection criteria to determine whether the tool is of interest to the Console community. The more of these questions we can answer positively, the more likely a tool is to be featured:

  • Is this interesting and useful to developers?
  • Is there a self-service signup?
  • Would this form part of a daily-use set of developer tools?
  • Does it make me a better developer?
  • Does it solve a problem I have experienced? How well does it solve it?
  • Would this be used by advanced, power-users?
  • Is the tool high quality?
  • Does it do the job it claims?
  • Does it have a good graphical and/or command line interface? Shortcuts? Accessibility?
  • Is it being actively maintained with regular bug fixes and updates?
  • Does it have good documentation? Is it fast?
  • Does it have any negative impact on security or privacy?
  • How would I feel recommending this to my friends? Would I personally tweet a link to it?

We try to be objective but we recognise that there is always a subjective element to deciding what we think is the best, most interesting set of tools each week.

In the future we plan to financially support Console through vendor relationships. While we will inevitably be reimbursed for offering exposure to certain companies within the Console newsletter, our editorial process will remain independent from our vendor partnerships. We will not accept payment for product inclusion.

As Console grows we're thinking about how to open up the editorial process to more developers. Both expanding our editorial team in-house and working with the wider developer community to feed into the selection process. We look forward to sharing more info on these developments later this year.

We regularly get asked when we will open up an archive of all the tools that have featured in the weekly newsletter. In the coming months, we plan to launch a new Developer Tools section on the Console website similar to Beta Console where developers can access the full archive of Console picks. This will provide a growing resource of interesting developer tools for immediate access.

In the meantime if you’re working on a new tool you’d like us to consider, or simply have a tool you’d like to recommend, then let us know!

Discover the best tools for developers

Console Newsletter - A free weekly email digest of the best tools and beta releases for developers. Every Thursday. See the latest email.