How To Optimize Cloud Firestore Usage

Sometimes we are using Cloud Firestore when we want to kick off the project fast. You don’t have to define your database schema, create tables, run migrations and so on. When we add cloud functions we are able to deploy it easily and it’s like a snap of your fingers.

Before you begin with Cloud Firestore let’s think about your project. What is it about? What are your traffic expectations? Do you need “special” data filtering? How much data do you expect in your database and what do you plan to do with them?

If you look at the pricing you can see your “unthought” implementation could be quite expensive. Some of the relational database functionalities may be missing and you would not be able to implement some features on the database side.

Today I would like to mention sharding method, live read and pagination. Why to use them, when to use them and how to use them to improve and optimize your Cloud Firestore usage.

Sharding

Imagine your Cloud Firestore collection has 20k documents. You want to show all of these documents to the users of your app. On the app side there is implemented the infinite scroll where data are shown. Naive and the fastest implementation is to read from the collection for example 20 documents per page and show them in the list.

If you look at the billing, one customer who will read through the whole list (all documents) means 20k reads. So how many customers do you have? How important are these data for your customers? This could be a really expensive implementation.

We can use sharding method. We will create a new collection. Each document will have just an array with the data arranged by 200 pieces of the starting collection. It means in the new collection there will be 100 documents with 200 long arrays of data from the previous collection. It’s like a pagination. In the first document there will be 1–200 documents = 1st page. In the second document there will be 201–400 other documents = 2nd page etc.

Then you will show data from the “shard” collection in your app. Improvement? 20k / 200 = 100 reads.

Live Read

Live read = when a document is changed, the document will be automatically refreshed on all devices. It means all devices are showing live (real time) document data.

Pricing is the same. When you are reading one document = 1 read.

Now imagine you are showing a number of the registered users on your homepage. Each new user will increase some counter that you are listening to. After each change it will “refresh” the number on your homepage. But how many times are you reading? How many users are on your homepage? How many users are completing their registration process? It could be a lot of reads.

Imagine there are 20k people on your homepage and 250 other people are completing their registration process. Each submitted registration will change the number of registered users on the homepage for each of the 20k users. Result? 20k * 250 = 5mil reads.

I recommend to use this functionality carefully. Is it really necessary for your use case? You can refresh the number each 60s for example or do some other improvements.

Pagination

I think most of us will imagine the limit and offset when we are talking about the pagination. But forget it when we are talking about it in the Cloud Firestore context! You have to use cursors!

Cloud Firestore allows you to use limit and offset to paginate your documents but it will cost something. When you use the offset in your query you will pay for all skipped documents also. It means when you want to access the second page with limit=10 and offset=10 you will pay for 10 found documents + 10 skipped documents!

When you are implementing the pagination use cursors — startAt, startAfter, endAt and endBefore methods.

Conclusion

Before you use Cloud Firestore please read the pricing to get to know what you will be charged for and think about your use cases. Use the sharding method for rapidly increasing collections, think about live read carefully and use cursors instead of limit and offset.