“Programs must be written for people to read, and only incidentally for machines to execute” — Harold Abelson

Here at Thumbtack, we code review every change that we merge. Code reviews have a lot of advantages: they reduce bugs and security issues, they lead to more readable code, and they help share context on code. They can even be useful for mentorship by serving as hands-on teaching about best practices.

Done right, code reviews can more than pay back the time they cost. Done wrong, they devolve into endless arguments about minutiae of the style guide. Over the course of nearly 100,000 code reviews, we’ve learned a bit about how to make them effective. To get the most out of the time code reviews take away from coding, try to find the balance between fixing the important issues and polishing mild imperfections. Thinking about possible security holes might take a long time, but is worth it. Finding lines wrapped in the wrong style is easy, but low value.

As the Author of a Code Review

The efficiency and effectiveness of the code review process depends as much on the author of the code as on the code reviewers. You won’t get much out of a code review for your change if you ask for a review from someone who doesn’t understand the codebase.

Before sending code out for review

Don’t use code reviews as the first point of communication about larger or more important changes. Especially with larger changes, discuss the purpose and design of the change with the team first so that you don’t get comments of the form, “this is a bad idea; let’s not do this” or “this should be rewritten and structured in a different way.” An ounce of communication can save a pound of wasted coding.

Depending on how your team works, this communication might look like a quick memo, a document describing the design, or even an in-person meeting. I’m a fan of using a document to host the design and discussion since it avoids yet another dreaded meeting and it serves as documentation of the discussion for understanding the purpose of the change down the road.

Another useful step to take before sending a change for review is to give it a quick review yourself. Often, you’ll be able to find the same things that other reviewers would. This allows you to fix them right away, without waiting for another feedback cycle in the code review process. It saves both you and your reviewers time. Likewise, also run unit tests and linters before sending out a change, for similar reasons.

How to send out a code review

The selection of reviewers is important. Always include someone who has enough understanding of the code and the relevant best practices to give it a thorough review. This might require including multiple people on the review, e.g. if it spans both backend and frontend components. It can also be useful to include additional people as reviewers so that they gain experience reviewing or so that they are aware of the changes in the code.

Let the reviewers you’ve selected know what you want from them. Are they CC’d so that they are aware of the change, or do you want them to be the primary reviewer? As a reviewer, do you want me to look at the whole change, or just a certain piece? If your code review tool automatically adds reviewers to the change, it can help to elect one of them as the “primary reviewer” and ask them to be the one to actually sign off on the change.

If you need a code review from someone who may not realize it (or need a fast review for some reason), it’s typically fine to private message that person.

How to respond to comments

Remember that comments are only intended to make the code better, and are not a personal attack (though it can certainly feel that way sometimes!). Often the busy reviewers will be very terse with their comments – that doesn’t mean they are grumpy, it just means they are trying to be efficient. Some reviewers use various bits of shorthand in their comments. For example, starting a comment with “nit:” indicates that the comment is a minor style “nitpick.” A comment saying just “100ch” means that the line is longer than 100 characters and should be wrapped.

Once you’ve gotten feedback on your change, there are a few tactical things you can do to make the process smooth and efficient:

As you make changes in response to comments, mark the comment as “done.” This is helpful for you to keep track of which comments you have and haven’t addressed yet (especially if you get interrupted), and it’s also helpful for the reviewer to see whether a comment has been addressed yet or not. If you aren’t making a change as requested, be sure to explain why in a response to the comment.

For each requested change, look at the rest of your code and see if the same change needs to be made elsewhere. To save time, reviewers often only mention the first instance of an issue they notice (instead of copying the same comment to each location in the code).

If a code review starts to involve a lot of back-and-forth cycles, it may be well worth your time (and the reviewer’s) to simply grab a room to hash out what the code should look like. There is a big difference between a response time of seconds and hours!

As the Reviewer

When you are reviewing code, always remember to be kind. You’re critiquing a person’s craft, which is tied to their identity. What seems like an objective observation to you may feel like a personal attack on the code author if poorly phrased.

Keep in mind that there is a ROI tradeoff to be made here. The goal of code reviews is to ensure the team is pushing good code, not to ensure all code pushed is perfect. Weigh the changes you ask for against their value, remembering that perfect is the enemy of good. Think about whether the time spent perfecting the code is the highest leverage activity available.

It’s useful to know the author when reviewing code. Are they a junior engineer fresh out of college or a seasoned expert in this area? Read code more carefully when you know an area is new to the author than when you know the author is experienced with that system. When reviewing a more junior engineer’s code (and when it’s appropriate and useful), use your comments as a means to mentor the author, not just as a means to bettering this specific code (teach a man to fish…) This makes communicating the general principles behind your suggestions as important as the suggestions themselves.

Be sure to be timely with responding to code reviews. The practice of reviewing code shouldn’t come at the cost of massively slowing down how quickly you can get changes out the door.

Do regular batches of reviews a few times a day. Doing too many at once will make your reviews less effective (you’ll get cross-eyed after an hour). Plus, it’s useful to get a few response cycles in over the course of a day. However, you need to balance responding quickly with getting distracted from other work. Personally, I find it useful to do code reviews when I’m already distracted (e.g. after lunch or a meeting).

It’s OK to ask for some changes to be made in follow-up commits for the sake of getting the current change out faster. Some issues (like missing unit tests) shouldn’t be deferred to future code reviews, but less critical changes like additional features or cleaning up more code are good candidates to do as a follow-up change.

Practical Notes

In practice, it’s useful to skip code reviews when reverting changes that have already been pushed. This allows engineers to speedily fix broken builds and production issues without waiting for code review. It’s also useful to define a set of criteria for when an emergency situation warrants pushing code without code review. For example, maybe it’s OK to push an emergency fix when a page is failing to render half the time, but not when a warning is logged every few minutes. The point is not exactly where you draw the line; the point is that you should have some line drawn.

When a code review has multiple reviewers, one of the reviewers will often want to indicate that they don’t have any changes they want to request but that they still want someone else to check over the code. We use a “+1” and “+2” system for that here – giving a code review a “+2” means that it can be merged, whereas a giving “+1” just means that they don’t object. This is especially useful when there are different reviewers for different parts of the review (e.g. one reviewer to look at the SQL, and another to look at the CSS). The first to review their section gives it a “+1”, and the second reviewer (seeing that the first reviewer already approved their piece) gives it a “+2”. Race conditions can cause duplicate “+1″s, but it is easily resolved by the author of the review bugging a reviewer to “+2”.

We also allow blocking submissions of changes as a safeguard against accidentally pushing something broken. By responding with a “-1”, a reviewer can block the code review from being submitted until they remove their “-1” (even if the code review has a “+2”). Interestingly, the most common use of this feature at Thumbtack is by the authors of the code reviews. They apply a “-1” to their own change to remind themselves to not merge until some other event (e.g. a database migration) happens.

Often, a few people get asked to review a disproportionate portion of the code reviews. If you are one of those people, feel free to ask the authors of some of the code reviews to pick different reviewers. It feels like it might be rude, but it ends up being better for both you (fewer things to review) and the author of the review (they get a response faster).

It’s also a good idea to let the author know if you don’t feel knowledgeable enough to review the change.

Where possible, use automated tooling to look for issues instead of doing it by hand. Automated checks like linters are better than manual checks in two ways: first, they save you time as the reviewer, and second, they never miss the issues they look for. For Go projects, making the build fail if the code isn’t formatted according to gofmt means that there is almost never any time wasted on discussing what the correct style is. No one tries to argue with a CI server. If there is any dangerous mistake that is easy to make (such as using a string function that is not multibyte-safe), consider finding a linter or writing a custom script (we use both) to look for those errors. They tend to be very easy to miss as human reviewers.

What To Look For

It’s helpful to have a checklist of what to look for in a code review because it’s really easy to forget to think about something important, especially when different code reviews force you to context-switch between different areas. Very little of this checklist should be applied to any single code review (unless it is quite a large change), but some subset applies to nearly every code review.

This list applies to both the reviewer and the author of the review.

The things you need to think about include whether the code is correct, whether the code has a good design, and whether it fits into the bigger picture of what is going on with the organization.

Is this the right change?

The first question, before diving into the code itself, is whether this change is even the right thing to be doing. Hopefully this issue got resolved by discussions before the code review, but it doesn’t always.

Ask yourself:

• Does this change need to happen at all? This question is more pertinent to the author than the reviewer, but it is still good for the reviewer to think through it. If (as the reviewer) you can’t tell, spend some time understanding the purpose of the change before reviewing it.
• What other ways of accomplishing the same thing are there? Is there a better way to do the same thing? Is writing code the right way to solve this problem? There are some classes of problems that can be solved by code, but which have better/cheaper solutions.

Cosmetic Issues

Of all the things to check in a code review, cosmetic issues are both the easiest to see and (usually) the lowest value to fix.

Ask yourself:

• Does it adhere to the style guide for the language?
• Is there a nicer way to express certain details in the code that would make them cleaner or more idiomatic? Idioms often exist for a good reason.
• Do any doc comments need to be updated with the new set of arguments and their types?
• If there are TODOs in the code, do they explain what needs to be done? Should that thing be done now?
• Are there any misspellings?
• Do the function and variable names make sense?
• Does the commit message explain why the change is being made? Would I understand the explanation a year from now? Is there any documentation or Jira ticket it could link to?

Functionality and Correctness

Depending on what the code does, there can be a lot of different areas to think through here.

General Correctness

Ask yourself:

• Does the code actually do what it claims? Is what the code should do defined well enough to decide that?
• Does it do the right thing in the face of invalid input, errors, timeouts, or other corner cases?
• Do any clients of an interface make assumptions that no longer hold? If the contract of an interface changes, all the code that depends on it need to be checked that it adheres to the new contract. (The contract of an interface means more than just the name of functions and data type of the parameters, it also include the behavior and side effects of the functions.)
• Do the new performance characteristics of this code work for all the callers?
• Will this code be too expensive (either in computer time or actual money!) to run?
• If an operation fails partway through, does it leave the system in a state where the operation can safely be retried?

Security

Being secure is a big part of being functionally correct!

Ask yourself:

• Are any passwords or authentication tokens checked in to the code? (Don’t do that!)
• Is access to read/write data limited to the right set of clients?
• Is there server-side validation in addition to client-side?
• Could the code be vulnerable to common attack vectors, such as cross-site scripting or timing attacks?
• Is there any possibility of accidentally logging a password?
• Are passwords stored as anything other than a strong (e.g. bcrypt) hash?
• Are there any inputs that would break a given function? If so, are the inputs that break things reasonable to expect?
• Are any very costly operations exposed to clients, allowing them to cause a large load on the server?
• Does anything prevent a client from trying to brute-force passwords?

This is by no means a complete set of things to think about in terms of security, but it is certainly better to think through an incomplete list than no list at all!

Systems with Concurrency

Concurrency involves its own interesting set of pitfalls to look for.

Ask yourself:

• Is this code safe to call concurrently, or does it make concurrent calls to things which are not safe to call concurrently?
• Are appropriate locks taken? Could a race condition cause an unlock before lock?
• Can it enter a deadlock or livelock?
• Is data access atomic?
• Is the amount of resources (e.g. threads, db connections) it can use limited or unbounded?

Again, this list isn’t trying to be exhaustive. Just remember to think about whether the logic is still correct given concurrency.

Asynchronous Work

If you do some work asynchronously (perhaps you process jobs in an SQS queue), these questions will probably sound familiar to you.

Ask yourself:

• Are errors in processing handled? Would I find out if all work started failing?
• Is there a way to find out how long it takes to process a piece of work? Is there a way to see the current size of the queue?
• Is failed work retried? Should it be?
• Are there limits to the number of retries? What happens when it fails too many times?

Network Calls

Ask yourself:

• Does this code respond appropriately if the system it calls is down? Can it offer a degraded experience instead of failing completely?
• Is there a timeout? Is that timeout shorter than the time in which callers of this code expect it to respond?
• Should there be retries? If there are retries, should they use exponential backoff?
• Do you need a circuit breaker to turn off traffic to a broken service?

HTTP Servers

Ask yourself:

• Does it have a shutdown timeout longer than the max time allowed for operations?
• Does it have client timeouts and request size limits?
• Does it respond with the right HTTP statuses? Does it ever do something silly like return a 200 status when there is actually an error?
• Does it return appropriate error messages for the caller? Make sure services don’t return internal details to a 3rd party, but also make sure that there is enough detail for other internal clients to deal with errors appropriately.
• Should it serve data over TLS?

Use of a database

Ask yourself:

• For batch jobs, is the rate of writes limited to avoid overwhelming the database system?
• If the database can rate-limit your request, do you back off and try again?
• (Assuming there are read-only database replicas) Is the code sensitive to replication lag? Does it handle larger-than-normal amounts of lag appropriately?
• When the change introduces a new data store, are there backups?
• Does the code make sure to maintain the integrity of the data?

If performance is of any concern (it isn’t always):

• Will the queries be efficient?
• Do they make use of an index to find the data?
• Is the number of rows the database engine has to scan to find the result set limited to a reasonable number?
• Is the size of the result set limited to a reasonable number?
• Are there any N+1 query patterns that could be fixed with a batch query?
• Are there cases of the same object being loaded twice?
• Are queries performant enough for the case with the largest data set? Think about the most active user on the site – you definitely want to make sure they get a good experience, but they often are working with the largest data set.
• Are any locks held for more than a few milliseconds?

Language-specific Issues

There are a lot of pitfalls that only apply to one language (or a certain class of languages, such as JVM languages). As both an author and a reviewer of code, it’s important to be aware of these.

If you write in Go, ask yourself:

• Does it send to/receive from a nil channel?
• Does it attempt to write to a nil map? This is especially important to check when adding a new field to a structure containing a map. Does everything that creates that structure initialize the map?
• Does it recover from panics in goroutines? (These kill the entire process unless handled.)
• Does it suffer frompointer aliasing in loops?

Design, Quality, and Clarity of the Code

A library-worth of books have been written about how to design code, so of course this list won’t capture everything.

Ask yourself:

• Can and should this change be split into smaller changes?
• Does the structure of the code make sense? Are concerns separated?
• Are there good interfaces in the right places? Are those interfaces well designed?
• Does the overall design permit a performant implementation?
• Is there any premature optimization or premature generalization?
• Does it reinvent any wheels (e.g. logging format, caching system) that don’t need to be reinvented?
• Does anything about this make future development difficult?
• Does the new code make use of interfaces that weren’t intended to be used/be used this way?
• Does this make use of the current latest practices, or does it follow outdated patterns or use deprecated libraries?
• Does the new code align with the structure of the existing code? Is it in the right namespace?
• Does the design permit effective testing, and is the code well tested? Are there integration tests between the callers and this code?
• Are any comments needed in the new or existing code explaining non-obvious details of the interaction of the code?
• How hard will this be to delete? This is one way to test if there is too much coupling.
• Is there enough tracing information in place to understand what is going on when something goes wrong (e.g. when the system starts being too slow)?
• If something goes wrong, is this change safe to rollback, or does it create state that makes that unsafe to do? Is the change safe to canary? For example, don’t add an endpoint and something that calls that endpoint in the same deploy.

Bigger Picture

One of the great advantages of code review is that the reviewer might have context that you don’t.

Ask yourself:

• Are there any conflicts with other upcoming or in-progress changes?
• Should any documentation be written or updated as a result of this change? This includes not just developer-facing documentation, but also documentation for the rest of the organization and even public-facing documentation (such as help pages).
• Do any people need to be notified about this change? How much time do they need to be given before the change is deployed?
• Are we solving the main problem, or is this a work-around for a deeper problem?
• Should some refactoring be done before making this change?
• Is the level of structure and polish appropriate for the purpose of this code? Does this spend a lot of code building up abstractions when it’s throw-away code, or should we invest more time building a good structure for it since this will probably be around for a long time? Does this look like “temporary” code that might end up actually living for a long time? If you’ll excuse a brief moment of soap-boxing: changing code later is often much, much harder than you think, so it’s almost always best to start off with a structure that doesn’t have any major problems. I don’t mean that more structure is better; I mean that you should avoid knowingly committing design sins with the intention of atoning for them later.

Conclusion

If you don’t do code review yet, I recommend that you try it. If you do practice code review, spend some time thinking about what could be improved about your process. Is it giving you the return on your investment of time that you want?

Are interested in engineering best practices? Join us and help us make our team even better!