Cloaked Search is a proxy for Elasticsearch/OpenSearch that protects the indexed data from prying eyes. Cloaked Search's API is the same as the underlying search service API.
In about 5 minutes, you will have:
- Elasticsearch/OpenSearch running on your local machine
- Cloaked Search running on your local machine
- sample data indexed with
summary
andbody
as protected fields - query results from sample queries using the protected
summary
andbody
fields
All stored data is on a temporary volume inside of docker. No changes will be made to your machine outside the directory where you cloned the try-cloaked-search
repo.
To try Cloaked Search you just need a basic *nix installation and docker
+ docker-compose
. Some of the commands below also use jq
for JSON formatting. If you don't have jq
, you can safely remove those portions of the command.
Clone the try-cloaked-search git repo.
git clone https://github.com/IronCoreLabs/try-cloaked-search.git
All other commands are assumed be run from within directory where you cloned the repo.
Note: This example install uses ports 9200 (Elasticsearch/OpenSearch) and 8675 (Cloaked Search). Be sure you don't have an existing search service running on port 9200 before beginning.
docker-compose -f elasticsearch/docker-compose.yml up # for Elasticsearch
or
docker-compose -f open-search/docker-compose.yml up # for Open Search
Note: Future commands will be targeting Cloaked Search on port 8675
try-cloaked-search
includes some test data. Since Cloaked Search uses a different key per (tenant, index, field),
documents with protected fields must be tagged with the tenant to which they belong.
Half of the articles in the test dataset are associated with tenant-1
, and the other half are associated with tenant-2
.
To better understand Cloaked Search's key management, refer to the configuration documentation.
./populate_index.sh
Note that all queries made with ./query-search-service.sh
are being made directly to your search service. We are using a script to detect if you're running OpenSearch or Elasticsearch, but there is no involvement from Cloaked Search. Since these requests go directly to the search service (port 9200), we can see what's actually stored.
Let's get all the documents belonging to tenant-1
and see what's in the index!
./query-search-service.sh '+tenant_id.keyword:"tenant-1"' | jq
We are protecting the body
and summary
fields from the original document. These fields are no longer attached to the document;
instead, the blind index tokens for body
are stored in the _icl_p_body
. Because we enabled index_prefixes
for the summary
field,
it has been translated to _icl_p_summary._value
and _icl_p_summary._index_prefix
.
You will also notice an _icl_encrypted_source
field; this contains an encrypted version of the fields that have been protected, which allows Cloaked Search
to return them to their original versions.
"_icl_p_summary": "7b76c95a 616544a2 b41fa81e 85933317 e30236d5 ...",
These are a couple examples of simple term queries. They are still querying on the title
field, which is unprotected.
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND title:Japan' | jq
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND title:cup' | jq
Compare these results to the ones returned by querying the search service directly. The same documents are returned, but the contents are very different.
You can see how Cloaked Search transparently handles the decryption of the document to allow you to see the data in the fields that were protected, summary
and body
.
./query-search-service.sh '+tenant_id.keyword:"tenant-1" AND title:Japan' | jq
./query-search-service.sh '+tenant_id.keyword:"tenant-1" AND title:cup' | jq
Now try querying on a protected field:
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND summary:glasgow' | jq
Queries can also be combined with ORs or ANDs, and you can mix protected and unprotected fields. For example,
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND (title:cup OR title:Japan)' | jq
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND (summary:cup OR title:Japan)' | jq
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND (summary:cup OR body:Japan)' | jq
Phrases can also be searched using quoted queries like this:
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND summary:"Cheerleading in Japan"' | jq
Finally, here is an example of a prefix query on the summary
field (the field with index_prefixes
enabled):
./query-cloaked-search.sh '+tenant_id.keyword:"tenant-1" AND summary:pro*' | jq
You can replace the query with anything you like. Make sure you have the +tenant_id.keyword
in the query. populate_index.sh
loaded 1000 documents, half are tagged with tenant-1
and the others are tagged with tenant-2
.
Cloaked Search also supports requests other than Query String queries. For example,
./query-cloaked-search-json.sh 'japan' 'tenant-1' | jq
This searches for Japan
in the protected summary
field over all the documents belonging to tenant-1
.
You will want to try out Cloaked Search on some of your own data in a more real environment.
Use the Kubernetes template in this repository to make a simple Kubernetes deployment.
See the configuration docs for info on how to configure and deploy Cloaked Search.
If you are interested in the underlying technology or in the security of the underlying search service index, see the What Is Encrypted Search page.