inanzzz | Reindexing elasticsearch index without downtime in symfony

Elasticsearch provides us Index Alias feature to prevent downtime. When our application uses index alias, it doesn't actually directly use the real index. It is like a symlink to the real index. When we create a new index, alias gets pointed to the new index and the old one gets deleted so the index alias always provides us data. The example uses FOSElasticaBundle.

Main benefits

You can update data in database and run reindexing.

You can update index mapping in your application and run reindexing.

Commands

# List all indexes
$ curl 127.0.0.1:9200/_cat/indices?v

# Recreate new index with new set of data
$ bin/console fos:elastica:populate --index=country

config.yml

We make use of index alias by adding use_alias: true config below. Depending on the application environment, our index alias name will be country_dev (app_dev.php/DEV), country_test (app_test.php/TEST), country (app.php/PROD) so on. You can ignore the rest of the config. We will be working on DEV environment in this example so expect to see country_dev as our index alias.

fos_elastica:
    clients:
        default: { host: 127.0.0.1, port: 9200 }
    indexes:
        country:
            client: default
            use_alias: true
            index_name: country_%kernel.environment%
            types:
                country:
                    properties:
                        id:
                            type: integer
                        name:
                            type: text
                        abbreviation:
                            type: text
                    persistence:
                        driver: orm
                        model: AppBundle\Entity\Country
                        finder: ~
                        provider: ~
                        listener: ~

We haven't created the index yet so terminal command should give us output below.

$ curl 127.0.0.1:9200/_cat/indices?v

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

Test

Create index first time

$ bin/console fos:elastica:populate --index=country # "country" is the index name from mapping above

Resetting country
 12/12 [============================] 100%
Populating country/country
Refreshing country

List indexes

As you can see below, the real index name is country_dev_2018-02-24-093619 which is what our index alias country_dev will talk to.

$ curl 127.0.0.1:9200/_cat/indices?v

health status index                         uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   country_dev_2018-02-24-093619 pLjMusDPQcGjC0EP56AxNQ   5   1         12            0     18.7kb         18.7kb

Recreate index

$ bin/console fos:elastica:populate --index=country # "country" is the index name from mapping above

Resetting country
 12/12 [============================] 100%
Populating country/country
Refreshing country

List indexes

As you can see below, the old index country_dev_2018-02-24-093619 has been replaced with the new one country_dev_2018-02-24-101126.

$ curl 127.0.0.1:9200/_cat/indices?v

health status index                         uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   country_dev_2018-02-24-101126 -HJQb-yjRhCZqpIR6tGL4w   5   1         12            0      4.6kb          4.6kb