Redis Data Handling and gRPC-JSON Conversion

Redis: Insertion, Indexing & Query

InsertionIndex CreationQuery
JSON.SET <key> $ '<JSON data>'FT.CREATE <index_name> ON JSON PREFIX 1 <key_prefix> SCHEMA <field_path> AS <field_alias> <data_type>FT.SEARCH <index_name> <query>
JSON.SET user:1 $ '{"name": "Alice", "age": 30, "skills": ["Python", "Redis"]}'FT.CREATE idx:user ON JSON PREFIX 1 user: SCHEMA $.name AS name TEXT $.age AS age NUMERIC
  • FT.SEARCH idx:user "@name:Alice"
  • FT.SEARCH idx:user "@age:[25 +inf]"
  • FT.SEARCH idx:user "@name:Alice" RETURN 2 name age

Explanation:

  • user:1 is the key.
  • $ refers to the root of the JSON document.

Explanation:

  • idx:user: Index name.
  • user:: Prefix for keys to index.
  • $.name AS name TEXT: Index name field as text.
  • $.age AS age NUMERIC: Index age field as numeric.

Explanation:

  • @name:Alice: Search for documents where name is “Alice”.
  • @age:[25 +inf]: Search for age greater than or equal to 25.
  • RETURN 2 name age: Return only name and age fields.

Protobuf to JSON/JSON to Protobuf

gRPC/JSONJSON/gRPC

Sample gRPC Framework:

syntax = "proto3";

message Person {
int32 id = 1;
string name = 2;
string email = 3;
bool isActive = 4;
repeated string skills = 5;
Address address = 6;
}

message Address {
string city = 1;
string postalCode = 2;
}

Sample JSON Model:

{

"id": 123,

"name": "Alice",

"email": "alice@example.com",

"isActive": true,

"skills": ["Python", "gRPC"],

"address": { "city": "Montreal", "postalCode": "H3A 1B1" }

}

Steps to Convert gRPC Framework to JSON

  1. Create a sample gRPC message:

    Person {

    id: 123

    name: "Alice"

    email: "alice@example.com"

    isActive: true

    skills: "Python"

    skills: "gRPC"

    address {

    city: "Montreal"

    postalCode: "H3A 1B1"

    }

    }

  2. Convert to JSON:

    {

    "id": 123,

    "name": "Alice",

    "email": "alice@example.com",

    "isActive": true,

    "skills": ["Python", "gRPC"],

    "address": {

    "city": "Montreal",

    "postalCode": "H3A 1B1"

    }

    }

Steps to Convert JSON to gRPC Framework

  1. Identify each field and its type:

    KeyValueType
    id123int32
    name"Alice"string
    email"alice@example.com"string
    isActivetruebool
    skills["Python", "gRPC"]repeated string
    addressobjectmessage Address
  2. Define the .proto file based on the JSON fields:

    Complete person.proto File

    syntax = "proto3";

    message Person {

    int32 id = 1;

    string name = 2;

    string email = 3;

    bool isActive = 4;

    repeated string skills = 5;

    Address address = 6; // Nested message

    }

    message Address {

    string city = 1;

    string postalCode = 2;

    }

Explanation

  • int32 id = 1;: The id field is an integer, and 1 is the field number.
  • string name = 2;: The name field is a string.
  • bool isActive = 4;: The isActive field is a boolean.
  • repeated string skills = 5;: The skills field is an array of strings.
  • Address: A nested message to represent the address field.

Request/Response using gRPC

Step 1: Create the .proto FileStep 3: Implement the ServerStep 4: Implement the Client

File: user.proto

syntax = "proto3";

package user;

// Define the User service

service UserService {

// RPC method to get user details by user_id

rpc GetUser (UserRequest) returns (UserResponse);

}

// Request message

message UserRequest {

int32 user_id = 1;

}

// Response message

message UserResponse {

int32 user_id = 1;

string name = 2;

string email = 3;

bool is_active = 4;

}

Step 2: Generate gRPC Code

> python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. user.proto

This will generate two files:

  1. user_pb2.py – Contains the message classes.
  2. user_pb2_grpc.py – Contains the service class and stub.

File: server.py

import grpc

from concurrent import futures

import time

import user_pb2

import user_pb2_grpc

# Create a class to define the server functions

class UserServiceServicer(user_pb2_grpc.UserServiceServicer):

def GetUser(self, request, context):

print(f"Received request for user_id: {request.user_id}")

# Dummy data for demonstration

users = {

1: {"name": "Alice", "email": "alice@example.com", "is_active": True},

2: {"name": "Bob", "email": "bob@example.com", "is_active": False},

}

user_data = users.get(request.user_id, {"name": "Unknown", "email": "", "is_active": False})

# Create a response

return user_pb2.UserResponse(

user_id=request.user_id,

name=user_data["name"],

email=user_data["email"],

is_active=user_data["is_active"]

)

# Create and start the gRPC server

def serve():

server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))

user_pb2_grpc.add_UserServiceServicer_to_server(UserServiceServicer(), server)

server.add_insecure_port("[::]:50051")

server.start()

print("Server started on port 50051")

try:

while True:

time.sleep(86400) # Keep server running for 1 day

except KeyboardInterrupt:

server.stop(0)

if __name__ == "__main__":

serve()

import grpc

import user_pb2

import user_pb2_grpc

def run():

# Connect to the server

with grpc.insecure_channel("localhost:50051") as channel:

stub = user_pb2_grpc.UserServiceStub(channel)

# Create a request

user_id = int(input("Enter user ID: "))

request = user_pb2.UserRequest(user_id=user_id)

# Send the request and get the response

try:

response = stub.GetUser(request)

print(f"User ID: {response.user_id}")

print(f"Name: {response.name}")

print(f"Email: {response.email}")

print(f"Active: {response.is_active}")

except grpc.RpcError as e:

print(f"Error: {e.details()}")

if __name__ == "__main__":

run()