Back to blog index

Sending the new listings notifications

This past week I did a rewrite on the listings in order to support searching by bra measurements. This made it just natural to be able to get listings notifications by this criteria as well.

This presented a number of problems.

The existing criteria to store listings notifications was different from the criteria to make a search. This lead to some incompatibilities, such as being able to save one listing notification for many brands or explicitly related to a bra model.

So, that was going to need a database change to a new table that stored now the same search query that was made on the screen. With the same parameters. That wasn't too difficult. I decided to store these search parameters on a JSON attribute instead of explicit database columns, since there's a lot of them now because of the measurements man/mix fields.

Notification.query = {JSON search query}

Now I had to change the code that sent the notifications to understand this new way of saving.

The natural solution was to feed this saved search query into the exact same routine that did the search lookup. If the results included the new listing then I should send an email for this notification. Something like this:

new_listing = <<Listing the user just created>>
for each notification out of all the notifications:
  results = search_listings(notification.query)
  if new_listing in results:
    send_email

The problem with this approach is that it requires a full database search for each notification, we have potentially thousands of notifications and over a couple of thousand of listings. So I wanted to avoid this approach.

What I ended up doing was to read out only the brand from the .query object and store that as a database column. I know that a big percentage of the notifications on the site are bound by a brand so I can use this to filter out.

The second part of this solution was to write a parallel match routine that does not require a full database read. And actually since I have the search query and the listing in hand, I could just match those quickly!

new_listing = <<Listing the user just created>>
all_notifications = notifications_for_brand(new_listing.brand) + notifications_without_brand
for each notification in all_notifications:
  if notification.matches(new_listing):
    send_email

Doing this was a huge improvement since I could use the database now to quickly filter out most of the invalid notifications. On top of that I did some other optimizations given that I only need one matching notification per user, as soon as I have one match I can stop doing any more checks for that user and continue to the next.

I'm happy with the result as it does not hit hard on the database, the only downside is that I need to keep the listings search and the notification matching routines in sync.

September 23, 2014
comments powered by Disqus