Search/Elasticsearch/

How to add search functionality to a React and Django website using Elasticsearch?

Published on

Having a search feature on your website helps you increase the user engagement time on your site. And also it attracts more users to your site. Implementing the search features was a difficult and expensive job. Here, we will learn about adding search functionality to a React and Django website with detailed steps.

Introduction:

Elasticsearch:

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data for lightning-fast search, fine‑tuned relevancy, and powerful analytics that scale with ease.

Elasticsearch Index:

An Elasticsearch index is like a ‘database' in a relational database. It has a mapping that defines multiple types. An index is a logical namespace that maps to one or more primary shards and can have zero or more replica shards.

In this blog, you will learn how to create a simple book search application using React, Django, and Elasticsearch.

Search functionality is achieved by adding a simple search bar and submit button in React and retrieving data from Elasticsearch using APIs developed with Django.

Adding the data to Elasticsearch is a mandatory step to create any search application. In this blog, I’m uploading Books sample data ( book_name, author, published_at, rating, price, image_url, etc.,) in an excel file and pushing it into the elastic index.

Flow diagram:

Prerequisites :

Ubuntu Server
Basic Django project setup
Basic React.js project setup

Steps:

Step 1: Install Elasticsearch and Kibana

To install Elasticsearch and Kibana follow the steps mentioned in the below link.

Ref –How to configure Elasticsearch and Kibana setup

Step 2: Create an index to store your data.

In the Kibana dashboard, Go to Dev Tools

To create an index, please run the following query
1PUT <your_index_name> 2 3for example: 4PUT books

To verify the index created from the above step, Navigate to Kibana > Stack Management > Index Management > Indices

Step 3: Create an API key to interact with Elasticsearch using REST APIs.

In the Kibana dashboard, Navigate to Kibana > Stack Management > API keys > click Create API key

In the Name section, Enter the API key name of your choice (for example: search_api_key and click Create API key

Once the API Key is created, save it into a local text file for future purposes. This API Key is mandatory for Django to interact with Elasticsearch using REST Endpoints.

Step 4: Search Backend API creation using Django

We can use any REST API development framework like Flask, Express.js, Django, and Java Spring Boot to create the search application backend which will pull the data from Elasticsearch.

In this Blog, we are gonna use Django Framework to create backend API.

Create a Django project named search_django using the following command.

1django-admin startproject search_django

Navigate to the project directory which you have created in the above step and create an app named app_search using the following command.

1cd search_django
2python manage.py startapp app_search

Once the project and app are created, the project structure will look similar to the below screenshot.

Install the required python packages by executing the following command.
1pip install -r requirements.txt

A sample requirements.txt file can be found in the following git repository file.

elasticsearch_based_book_search_application/requirements.txt at main · episyche/elasticsearch_based_book_search_application 

Update the Django settings.py file with the following items.
- In the INSTALLED_APPS section add your app name (i.e. app_search)
- Append ELASTIC_URL, ELASTIC_INDEX, and ELASTIC_API_KEY(i.e. from Step 3: Create an API key to interact with Elasticsearch using REST APIs. ) at end of settings.py file to allow Django to access Elasticsearch.

1#settings.py
2
3INSTALLED_APPS = [
4    .....
5    'app_search'
6]
7.
8.
9.
10
11#elasticsearch
12ELASTIC_URL = <YOUR_ELASTICSEARCH_URL>
13ELASTIC_INDEX = <YOUR_INDEX_NAME>
14ELASTIC_API_KEY = <YOUR_ELASTIC_API_KEY>

A sample settings.py file can be found in the following git repository file. 

elasticsearch_based_book_search_application/settings.py at main · episyche/elasticsearch_based_book_search_application 

To push sample data to the elastic search index, In your Django project create a folder named scripts. Under the scripts folder create a file named push_data_to_elastic_index.py.

In push_data_to_elastic_index.py, add the following code and run it ( In this file reads an excel file that contains some sample data, converts it into JSON, and pushes that data in the Elasticsearch index )
Also update the YOUR_ELASTIC_URL, YOUR_ELASTIC_INDEX, and YOUR_ELASTIC_API_KEY(i.e. from Step 3: Create an API key to interact with Elasticsearch using REST APIs. ) in push_data_to_elastic_index.py file.
1import requests 2import openpyxl 3 4doc_url = 'https://<YOUR_ELASTIC_URL>/<YOUR_ELASTIC_INDEX>/_doc' 5 6headers = { "Authorization": "ApiKey <YOUR_ELASTIC_API_KEY>"} 7 8excel_file = openpyxl.load_workbook("static/files/books_list.xlsx") 9sheet = excel_file.active 10 11for i in range(2 , sheet.max_row+1): 12 dic = {} 13 book_name = sheet.cell(row=i , column=2).value 14 if book_name: 15 dic['book_name'] = sheet.cell(row=i , column=2).value 16 column_3 = sheet.cell(row=i , column=3).value 17 if column_3 is not None : 18 try: 19 split_string = column_3.split('|') 20 dic['author'] = split_string[0] 21 dic['published_at'] = split_string[1] 22 except: 23 pass 24 dic['rating'] = sheet.cell(row=i , column=4).value 25 dic['price'] = sheet.cell(row=i , column=6).value 26 dic['image_url'] = sheet.cell(row=i , column=9).value 27 dic['amazon_id'] = sheet.cell(row=i , column=8).value 28 dic['product_url'] = sheet.cell(row=i , column=1).value 29 try: 30 upload_resp = requests.post(doc_url, headers=headers, json=dic , verify=False) 31 print('upload res----' , upload_resp) 32 except Exception as err: 33 print('er----' , err)
A sample push_data_to_elastic_index.py file can be found in the following git repository file.

elasticsearch_based_book_search_application/push_data_to_elastic_index.py at main · episyche/elasticsearch_based_book_search_application 

Execute the above python script using the following commands.
1python3 push_data_to_elastic_index.py
To verify the data stored in your index, Please go to Kibana Dashboard and navigate to Dev Tools, Run the following query
1GET <your_index_name>/_search 2{ 3 "query": { 4 "match_all": {} 5 } 6}

Navigate to app_search django app directory, and create a Search class in views.py.
You can use any search query like the match (Match query | Elasticsearch Guide [8.5] | Elastic ), or query_string (Query string query | Elasticsearch Guide [8.5] | Elastic ). I m using a wildcard ( Wildcard query | Elasticsearch Guide [8.5] | Elastic )query to retrieve the data from elasticsearch.
1import requests 2from django.conf import settings 3from django.http import JsonResponse 4from rest_framework.views import APIView 5 6search_url = settings.ELASTIC_URL+ settings.ELASTIC_INDEX + "/_search/" 7headers = { "Authorization": "ApiKey "+ settings.ELASTIC_API_KEY} 8 9class Search(APIView): 10 def post(self , request): 11 data = request.data 12 keyword = data['keyword'] 13 search_query = { 14 "query": { 15 "wildcard": { 16 "book_name.keyword": { 17 "value" : "*"+ keyword +"*" 18 } 19 } 20 } 21 } 22 try: 23 search_response = requests.post(search_url, headers=headers, json=search_query , verify=False) 24 resp = search_response.json() 25 except Exception as er: 26 print('search err-----' , er) 27 28 value = resp['hits']['total']['value'] 29 print('---value---------' , value) 30 matched_data = [] 31 for hits_position in range(value): 32 objects = resp['hits']['hits'][hits_position]['_source'] 33 matched_data.append(objects) 34 35 return JsonResponse({'data' : matched_data})

A sample views.py file can be found in the following git repository file.

elasticsearch_based_book_search_application/views.py at main · episyche/elasticsearch_based_book_search_application 

After creating the class add the urls.py file under your app_search django app.

In your core folder navigate to urls.py(core folder) and add your path with <appname> and <url_filename>(i.e. created in the above step). Your core urls.py settings are like the below.

1from django.contrib import admin
2from django.urls import path , include
3
4urlpatterns = [
5    path('admin/', admin.site.urls),
6    path('search/', include('<your_app_name>.<url_filename>')),
7]

For example:

1from django.contrib import admin
2from django.urls import path , include
3
4urlpatterns = [
5    path('admin/', admin.site.urls),
6    path('search/', include('app_search.urls')),
7]

A sample urls.py(core folder) file can be found in the following git repository file.

elasticsearch_based_book_search_application/urls.py at main · episyche/elasticsearch_based_book_search_application 

Now, navigate to the Django app directory (i.e app_search) and add the Search Class API path in urls.py
1from app_search import views 2from django.urls import path 3from .views import * 4 5urlpatterns = [ 6 path('books/' , Search.as_view()), 7]

A sample urls.py(app_search) file can be found in the following git repository file.

elasticsearch_based_book_search_application/urls.py at main · episyche/elasticsearch_based_book_search_application 

To run the Django project . Run the following command
1python manage.py runserver
Test the API using postman or any other API Client tool.

Step 5: Search Web Page creation using React.

Run the following command to create a React project.
1npx create-react-app search_react
Navigate to the created project directory
1cd search_react
Under the src folder create a folder named components and create search.js
The final project structure is given below

In search.js add a text area and submit button to search data.
To fetch data from the backend API which we created in Step 4: Django Backend Creation( http://127.0.0.1:8000/search/books/ ) add the following function in your search.js

1  function GetBooks(body) {
2      var keyword = document.getElementById('keyword')
3      async function Put(value) {
4          fetch("http://127.0.0.1:8000/search/books/", {
5              method: "POST",
6              headers: { "Content-Type": "application/json", },
7              body: JSON.stringify({
8                  "keyword": value
9              })
10          })
11              .then((res) => res.json())
12              .then((response) => {
13                  setData(response.data)
14              })
15              .catch((err) => console.error("Error: ", err))
16      }
17      Put(keyword.value)
18  }

Call the above function when you click the submit button
1 <div className='search-container'> 2 <textarea id="keyword"></textarea> 3 <br/> 4 <button onClick={(e) => { e.preventDefault(); GetBooks() }} type="submit" className="submit">Submit</button> 5</div>
When you click the submit button it calls the backend API. In the backend, it performs a search operation on Elasticsearch and fetches you the relevant information.
After completing the search operation you will get the matched data from Elasticsearch.
To view the data from Elasticsearch, add the below code in search.js

1 {
2    data.map((e) => (
3        <div key={e.amazon_id} className='result'>
4            <img src={e.image_url} width='120px' alt='book-img' className="book-img"></img>
5            <div className="book-details">
6                <h3>{e.book_name}</h3>
7                <div>
8                    <p>{e.price}</p>
9                    <p>{e.author}</p>
10                    <p>Published - {e.published_at}</p>
11                </div>
12            </div>
13        </div>
14    ))
15}

A sample Search.js file can be found in the following git repository file.

elasticsearch_based_book_search_application/search.js at main · episyche/elasticsearch_based_book_search_application 

Result:

After successfully running both the backend and frontend apps. If you open the react URL (i.e localhost:3000) in your browser the following page will appear.