Dev blog

New picture uploader

A common complaint that many people had for a long time was that uploading pictures was painfully slow, timing out in many occasions.

The problem was the way we handled them; and allowing to upload many files at once only made things N times slower. The processing for each of them was very naive.

Read more...

Accounts without a email

There was a mishap on the last update regarding the signup process on the site that affects users that authenticate using their Reddit accounts.

Reddit does not provide third party sites the email address the user has on file. I think this is a fine security policy. The problem is that in the old signup flow we'd always ask users for an email address. In the new update since most authentications providers would give an email address we opted to remove that screen completely forgetting about Reddit.

Read more...

Upgrade Django social auth because Google drops OpenID support

Google has announces that they will drop OpenID support on April 20th.

We use OpenID to authenticate users with their Google account, so this affects Bratabase directly. We have over 6000 accounts that use this login method.

Read more...

Database server upgrade

Just had to do a server upgrade, our database server went up from 2GB to 4GB in ram and while at it also increased the swap partition from 512 to 2GB. Also did an security update of the packages on the server that was needed.

This really made a difference on the database performance. I still need to do a lot of tuning to our Postgres database, but on its current state it was swapping like crazy which caused lots of deadlocks and very poor performance. Lots of simple queries were taking over a second to complete.

Read more...

Sending the new listings notifications

This past week I did a rewrite on the listings in order to support searching by bra measurements. This made it just natural to be able to get listings notifications by this criteria as well.

This presented a number of problems.

Read more...

This year's April's fools

It was 11 PM March 31 when I saw a tweet warning about April's fools and I realized that I had nothing prepared for the site so I needed to think of something quick to implement. My first idea was to turn the site black and white, but that would have required a lot of CSS and images editing to get going.

Then after some conversations about cup depth, the image of a face trying to get its nose to the deepest part of the cup came to my head and the idea of smelling the bra followed shortly after. It was something simple enough to implement.

Read more...

Multiple asynchronous workers

Many events on the site have repercussion beyond the direct object where they acted up on and executing this repercussion can take some time. If we waited for all this to happen before showing the next page to the user, the site would be extremely slow.

These are some examples of such activities:

Read more...

Faster model gallery pages

The last few weeks I've been working on improving the site's performance since beginning November last year it went to the floor.

There are still some pieces that I have not found completely but I spotted one of the biggest problems. It was template rendering. The Django views were returning too basic objects on the template's rendering context and too much logic was put on the templates, so they ended up doing many related queries (hidden in methods) or template tags.

Read more...

Loading menu items asynchronously

It always bugged me that in order to render any page where the user is logged in I had to do a number of database queries to be able to render the notifications and main menu. Specifically:

  • Unread messages
  • Unread notifications
  • Bra count and reputation on main menu

Read more...

Now we run on 3 servers

Today I added a new server to the site. I will call this the "Task" server, but the server's host name is "Perlea".

We were running with a two server setup App server(Ewa) and Database server(Kate).

The DB server is running a bit more than just the DB, it is also running the Elastic search server and now, a Redis server.

The new Redis server will be used for three purposes:

  • As a task queue, I was using RabbitMQ before to queue the asynchronous tasks, which was running on the application server.
  • As a Key/Value store for generated thumbnails cache. I was using a database table for this, but it was doing too many hits competing with other reads I needed.
  • Denormalized storage, I had a small Redis instance running on the application server which was doing some API counting and such, this same usage will not go to the Redis instance running on the DB server.

Read more...