Elasticsearch Index for the The Movie Database.
This repository is part of the Think Like a Relevancy Engineer training provided by OpenSource Connections.
- Download this repo
- Install the software (using either Docker or installing manually)
- Index the TMDB movie data
- Confirm Elasticsearch has the data
- Install Postman (optional)
Download the zip from https://github.com/o19s/es-tmdb/archive/master.zip, and you will get the file es-tmdb-master.zip. Unzip this file, resulting in the directory es-tmdb-master.
After you have this download, change into the newly created directory.
If you have Docker installed and running.
Make sure you have at least 4gb of memory available for Docker, the default is 2gb. See Docker's Preferences >>> Resources-tab, to adjust.
Linux/Windows/Mac:
docker-compose up
Give it a minute to fully boot up then use a browser to go to http://localhost:9200 and http://localhost:5601 to confirm Elasticsearch and Kibana are running.
- Download Elasticsearch 8.9.2
- Unzip to where you'd like to run Elasticsearch
- Add the following to config/elasticsearch.yml
http.cors.allow-origin: "/https?:\\/\\/(.*?\\.)?(quepid\\.com|splainer\\.io)/"
http.cors.enabled: true
indices.query.bool.max_clause_count: 10240
- Run Elasticsearch
bin/elasticsearch
-
In your browser, navigate to http://localhost:9200 to confirm Elasticsearch is running
-
Install the plugins for LTR and Querqy
bin/elasticsearch-plugin install -b "https://github.com/o19s/elasticsearch-learning-to-rank/releases/download/v1.5.8-es8.9.2/ltr-plugin-v1.5.8-es8.9.2.zip"
bin/elasticsearch-plugin install -b "https://repo1.maven.org/maven2/org/querqy/querqy-elasticsearch/1.7.es892.0/querqy-elasticsearch-1.7.es892.0.zip"
-
Download Kibana 8.9.2
-
Unzip to where you'd like to run Kibana
-
Run Kibana
bin/kibana
- In your browser, navigate to http://localhost:5601 to confirm Kibana is running with the Analyze plugin included
Once Elasticsearch and Kibana are ready go, we need to create our example search index.
Linux/Mac:
./index.sh
Windows:
powershell index.ps1
If you don't have admin permissions to run PowerShell scripts, then open up the PowerShell ISE application as an administrator. Then switch to the downloaded code directory and cut and paste the contents of index.ps1
into the console to work around this issue.
Run a wildcard search and confirm you get results.
Postman helps manage API requests. The examples from the TLRE slides exist here too as a Postman Collection (es-postman-collection.json
). We like using Postman because it makes tinkering with query parameters nicer.
We will mainly Kibana's DevTools for tinkering, but if you already familiar with Postman you can use that and get similar milage.
If you want to use Postman during the TLRE class:
- Download Postman for your OS
- Open Postman and Import (top-menu >> File)
es-postman-collection.json
- Define a global variable (grey eye icon in the upper-right)
es_host
to point to your running Elasticsearch instance (default islocalhost:9200
) - Tinker with the base URL, Params or JSON Body (optional)
- Press 'Send' (blue rectangle button right of URL bar) to Search!