According to the official documentation , this function is used to run a coroutine in an event loop on another thread, and it returns a Future object.

Based on the documentation, we can quickly write a toy example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

import asyncio
import threading

async def f():
    print("Inside coroutine f()")
    raise ValueError

def start_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

def main():
    loop = asyncio.new_event_loop()
    thread = threading.Thread(target=start_loop, args=(loop,))
    thread.start()

    future = asyncio.run_coroutine_threadsafe(f(), loop)

    try:
        future.result()
    except Exception as e:
        print("Caught exception:", repr(e))

    print("Stopping loop")
    loop.call_soon_threadsafe(loop.stop)
    thread.join()

if __name__ == "__main__":
    main()

The main function creates a thread and an event loop, then submits the coroutine f to that event loop using asyncio.run_coroutine_threadsafe. It then retrieves the result with future.result(), handles exceptions, and finally stops the event loop. Everything seems to work fine.

The output is:

Inside coroutine f()
Caught exception: ValueError()
Stopping loop

Now, here is the question: what happens if the coroutine f raises SystemExit instead of ValueError?

According to the official documentation , SystemExit inherits from BaseException rather than Exception. So would it be enough to change Exception to BaseException on line 21? You will find that after making these two changes and running the program again, it prints Inside coroutine f() and then hangs indefinitely.

We need to check the source code of asyncio.run_coroutine_threadsafe to understand what is happening. Here is the relevant part:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

def run_coroutine_threadsafe(coro, loop):
    """Submit a coroutine object to a given event loop.

    Return a concurrent.futures.Future to access the result.
    """
    if not coroutines.iscoroutine(coro):
        raise TypeError('A coroutine object is required')
    future = concurrent.futures.Future()

    def callback():
        try:
            futures._chain_future(ensure_future(coro, loop=loop), future)
        except (SystemExit, KeyboardInterrupt):
            raise
        except BaseException as exc:
            if future.set_running_or_notify_cancel():
                future.set_exception(exc)
            raise

    loop.call_soon_threadsafe(callback)
    return future

This function treats SystemExit and KeyboardInterrupt specially by directly raising them, while for other exceptions, it calls future.set_exception. Since it does not call set_exception for these two exceptions, future.result() never gets a result.

Here is a modified version of the original program:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

import time
import asyncio
import threading

async def f():
    print("Inside coroutine f()")
    raise SystemExit

def start_loop(loop):
    asyncio.set_event_loop(loop)
    loop.run_forever()

def main():
    loop = asyncio.new_event_loop()
    thread = threading.Thread(target=start_loop, args=(loop,))
    thread.start()

    future = asyncio.run_coroutine_threadsafe(f(), loop)

    try:
        future.result(timeout=3)
    except TimeoutError:
        print("Timeout")
        print("Future done?", future.done())
        print("Future cancelled?", future.cancelled())
        print("Loop alive?", loop.is_running())
        print("Thread alive?", thread.is_alive())
    except BaseException as e:
        print("Caught exception:", repr(e))

    print("Stopping loop")
    loop.call_soon_threadsafe(loop.stop)
    thread.join()

if __name__ == "__main__":
    main()

The output is:

Inside coroutine f()
Timeout
Future done? False
Future cancelled? False
Loop alive? False
Thread alive? False
Stopping loop
Task exception was never retrieved
future: <Task finished name='Task-1' coro=<f() done, defined at /home/mortalhappiness/test/test.py:5> exception=SystemExit()>
Traceback (most recent call last):
  File "/home/mortalhappiness/miniforge3/envs/test/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()
  File "/home/mortalhappiness/miniforge3/envs/test/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/home/mortalhappiness/test/test.py", line 11, in start_loop
    loop.run_forever()
  File "/home/mortalhappiness/miniforge3/envs/test/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/home/mortalhappiness/miniforge3/envs/test/lib/python3.12/asyncio/base_events.py", line 1986, in _run_once
    handle._run()
  File "/home/mortalhappiness/miniforge3/envs/test/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/home/mortalhappiness/test/test.py", line 7, in f
    raise SystemExit
SystemExit

This behavior is a bit tricky. Both future.done() and future.cancelled() return False, and calling future.result() or future.exception() will hang indefinitely without returning anything. However, if you do not call them, you will see a warning stating that a task exception was never retrieved. And you can see that both the event loop and the thread died.

Conclusion

If you use asyncio.run_coroutine_threadsafe to execute a coroutine that raises SystemExit or KeyboardInterrupt, calling future.result() or future.exception() will not work and will hang indefinitely. Additionally, the thread and event loop will terminate, but the main thread will remain alive.

client-go is the official Golang client for Kubernetes, responsible for interacting with the Kubernetes API server using REST API. In fact, client-go can do almost anything, not just for writing operators. Even the internal implementation of kubectl is based on client-go. As for more specialized frameworks used to write operators, including controller-runtime , kubebuilder , and operator-sdk , they will be introduced later in this series.

Introduction to Sample Controller Mechanism

sample-controller is an official Kubernetes example operator implemented using client-go.

To understand the code, we need to first understand how the operator we write interacts with client-go. The explanation here is a simplified version of the official documentation .

The above image comes from the official documentation and can also be found in many tutorials online.

The upper part of the image shows the internal components of client-go. It looks complicated, with terms like Reflector, Informer, and Indexer, but actually, you only need to understand Informer. The main function of Informer is to notify us when the status of resources changes. Why don’t we just hit the API directly to the Kubernetes API server? This is because calling the API is an expensive operation, and so the Informer maintains an Informer Cache to reduce the number of requests to the API server.

The lower part shows what we need to write ourselves:

• Resource Event Handlers: When Informer notifies us of a change in a resource’s status, we decide what to do, which usually means putting its key (namespace + name) into the workqueue.
• Workqueue: This stores the keys of all objects waiting to be processed. Our operator constantly retrieves items from the workqueue and tries to bring the cluster to the desired state. If it fails, the object key may need to be added back to the workqueue for further processing.

Sample Controller Codebase Walkthrough

Defining the CRD

In register.go , the GroupName is defined, and in v1alpha1/types.go , the type for the CRD is defined. You can see that it defines a Foo resource as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

// Foo is a specification for a Foo resource
type Foo struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   FooSpec   `json:"spec"`
	Status FooStatus `json:"status"`
}

// FooSpec is the spec for a Foo resource
type FooSpec struct {
	DeploymentName string `json:"deploymentName"`
	Replicas       *int32 `json:"replicas"`
}

// FooStatus is the status for a Foo resource
type FooStatus struct {
	AvailableReplicas int32 `json:"availableReplicas"`
}

Apart from the basic TypeMeta and ObjectMeta, it defines Spec and Status. Spec is where users can input data, defining the “desired state of the resource.” Status is where our Operator writes values, representing the “current state of the resource.”

The Sample Controller uses Kubernetes’ code-generator to generate typed clients, informers, listers, and deep-copy functions for the CRD. So whenever you modify types.go, you need to run ./hack/update-codegen.sh to regenerate the code.

Program Entrypoint

Next, look at main.go , which is the entry point of the program. It’s actually very simple, just pay attention to these lines:

1
2
3
4
5
6
7
8

kubeClient, err := kubernetes.NewForConfig(cfg)
exampleClient, err := clientset.NewForConfig(cfg)
kubeInformerFactory := kubeinformers.NewSharedInformerFactory(kubeClient, time.Second*30)
exampleInformerFactory := informers.NewSharedInformerFactory(exampleClient, time.Second*30)
controller := NewController(ctx, kubeClient, exampleClient,
    kubeInformerFactory.Apps().V1().Deployments(),
    exampleInformerFactory.Samplecontroller().V1alpha1().Foos())
controller.Run(ctx, 2)

Basically, it creates clients and informers for both Kubernetes built-in resources and our custom Foo resource, then passes them to NewController, and finally calls controller.Run.

Main Logic

Now, let’s examine the main part: controller.go .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

fooInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc: controller.enqueueFoo,
    UpdateFunc: func(old, new interface{}) {
        controller.enqueueFoo(new)
    },
})

deploymentInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
    AddFunc: controller.handleObject,
    UpdateFunc: func(old, new interface{}) {
        newDepl := new.(*appsv1.Deployment)
        oldDepl := old.(*appsv1.Deployment)
        if newDepl.ResourceVersion == oldDepl.ResourceVersion {
            // Periodic resync will send update events for all known Deployments.
            // Two different versions of the same Deployment will always have different RVs.
            return
        }
        controller.handleObject(new)
    },
    DeleteFunc: controller.handleObject,
})

This part shows the event handler we talked about earlier, where you can register AddFunc, UpdateFunc, and DeleteFunc. When the informer detects a change in the resource, it will call the corresponding function. You can see that for fooInformer, it simply calls enqueueFoo, while for deploymentInformer, it calls handleObject.

1
2
3
4
5
6
7
8

func (c *Controller) enqueueFoo(obj interface{}) {
	if objectRef, err := cache.ObjectToName(obj); err != nil {
		utilruntime.HandleError(err)
		return
	} else {
		c.workqueue.Add(objectRef)
	}
}

enqueueFoo is just adding the key of the Foo object to the workqueue. You can see here:

• cache.ObjectToName : Takes an object and converts it to ObjectName.
• ObjectName : This is just namespace + name.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

func (c *Controller) handleObject(obj interface{}) {
	...
	if ownerRef := metav1.GetControllerOf(object); ownerRef != nil {
		// If this object is not owned by a Foo, we should not do anything more
		// with it.
		if ownerRef.Kind != "Foo" {
			return
		}

		foo, err := c.foosLister.Foos(object.GetNamespace()).Get(ownerRef.Name)
		if err != nil {
			logger.V(4).Info("Ignore orphaned object", "object", klog.KObj(object), "foo", ownerRef.Name)
			return
		}

		c.enqueueFoo(foo)
		return
	}
}

This is part of the handleObject function. It checks whether the owner of the deployment is Foo. If it’s not, we ignore it. If it is, we add the corresponding Foo key to the workqueue. This relates to a concept called OwnerReference , where certain objects in Kubernetes are owned by others. The default behavior is that when the owner is deleted, the owned objects are also deleted. For example, a ReplicaSet is the owner of Pods, so when the ReplicaSet is deleted, the Pods it manages are also deleted. This is also why there is no DeleteFunc handler for fooInformer — when Foo is deleted, we want to delete all corresponding deployments, but since the owner of the deployment is already set to Foo, they will be deleted automatically without further handling.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

func (c *Controller) Run(ctx context.Context, workers int) error {
	...
	for i := 0; i < workers; i++ {
		go wait.UntilWithContext(ctx, c.runWorker, time.Second)
	}
	...
}

func (c *Controller) runWorker(ctx context.Context) {
	for c.processNextWorkItem(ctx) {
	}
}

Run is the entry point called by the controller in main.go. It starts multiple goroutines to run runWorker. runWorker is simply an infinite loop calling processNextWorkItem.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

func (c *Controller) processNextWorkItem(ctx context.Context) bool {
	objRef, shutdown := c.workqueue.Get()

	...

	// Run the syncHandler, passing it the structured reference to the object to be synced.
	err := c.syncHandler(ctx, objRef)
	if err == nil {
		c.workqueue.Forget(objRef)
		logger.Info("Successfully synced", "objectName", objRef)
		return true
	}
	utilruntime.HandleErrorWithContext(ctx, err, "Error syncing;

 requeuing for later retry", "objectReference", objRef)
	c.workqueue.AddRateLimited(objRef)
	return true
}

This is a portion of processNextWorkItem. First, it retrieves an object key from the workqueue, then calls syncHandler to handle it. If successful, it removes it from the workqueue. Otherwise, it performs error handling and puts the key back into the workqueue for later processing.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

func (c *Controller) syncHandler(ctx context.Context, objectRef cache.ObjectName) error {
	...

	// Get the Foo resource with this namespace/name
	foo, err := c.foosLister.Foos(objectRef.Namespace).Get(objectRef.Name)

	...

	deploymentName := foo.Spec.DeploymentName

	...

	// Get the deployment with the name specified in Foo.spec
	deployment, err := c.deploymentsLister.Deployments(foo.Namespace).Get(deploymentName)
	// If the resource doesn't exist, we'll create it
	if errors.IsNotFound(err) {
		deployment, err = c.kubeclientset.AppsV1().Deployments(foo.Namespace).Create(context.TODO(), newDeployment(foo), metav1.CreateOptions{FieldManager: FieldManager})
	}

	if err != nil {
		return err
	}

	// If the Deployment is not controlled by this Foo resource, we should log
	// a warning to the event recorder and return error msg.
	if !metav1.IsControlledBy(deployment, foo) {
		msg := fmt.Sprintf(MessageResourceExists, deployment.Name)
		c.recorder.Event(foo, corev1.EventTypeWarning, ErrResourceExists, msg)
		return fmt.Errorf("%s", msg)
	}

	// If this number of the replicas on the Foo resource is specified, and the
	// number does not equal the current desired replicas on the Deployment, we
	// should update the Deployment resource.
	if foo.Spec.Replicas != nil && *foo.Spec.Replicas != *deployment.Spec.Replicas {
		logger.V(4).Info("Update deployment resource", "currentReplicas", *foo.Spec.Replicas, "desiredReplicas", *deployment.Spec.Replicas)
		deployment, err = c.kubeclientset.AppsV1().Deployments(foo.Namespace).Update(context.TODO(), newDeployment(foo), metav1.UpdateOptions{FieldManager: FieldManager})
	}

	if err != nil {
		return err
	}

	// Finally, we update the status block of the Foo resource to reflect the
	// current state of the world
	err = c.updateFooStatus(foo, deployment)
	if err != nil {
		return err
	}

	c.recorder.Event(foo, corev1.EventTypeNormal, SuccessSynced, MessageResourceSynced)
	return nil
}

Finally, this is a portion of syncHandler. Here is where we write the actual logic, adjusting the cluster to match the desired state declared by the user in the Spec. The desired state in this case is that the deployment specified in Spec has been created and that the replica count matches what is declared in Spec.

Conclusion

After going through this, you may feel that I’ve only covered a small part of the Sample Controller code. That’s because client-go is a rather low-level library, and there are some downsides to using it for writing operators:

• We don’t need to write much custom logic, but still have to write some boilerplate, which can feel redundant.
• When watching to different resources, we need to declare informers, listers, and other repetitive things for each resource. For example, in the Sample Controller, both fooInformer and deploymentInformer are declared, and managing multiple resources becomes cumbersome.

These drawbacks have led to the development of other frameworks that are more specialized for writing operators, such as controller-runtime , kubebuilder , and operator-sdk . Stay tuned for future articles in this series to learn about these frameworks.

References

• https://github.com/kubernetes/sample-controller/blob/master/docs/controller-client-go.md
• https://www.linkedin.com/pulse/kubernetes-custom-controllers-part-1-kritik-sachdeva/
• https://www.linkedin.com/pulse/kubernetes-custom-controller-part-2-kritik-sachdeva/

If you just want to see the conclusion, skip to the Summary . Please note that the content discussed here is only guaranteed to work for version 0.3.2. Other versions have not been tested.

Introduction

GraphRAG is a new Retrieval-Augmented Generation (RAG) technology released by Microsoft. Unlike the usual RAG approach, which breaks text into chunks, vectorizes them, and stores them in a vector store, and then compares the similarity by vectorizing the user’s query, GraphRAG extracts information from text and builds a Knowledge Graph. This approach has the advantage of better understanding the complex relationships between different documents.

As of the time of writing, the latest official version is 0.3.2, and here is the GitHub repository link for version 0.3.2 . While the content discussed here might work for other versions, it is not guaranteed.

If you’ve read the official tutorial, you’ll notice that the config for version 0.3.2 only supports OpenAI and Azure OpenAI. You need to put your API Key in the config file to use it, which works and is the simplest approach. However, GraphRAG requires frequent LLM calls during indexing, which makes it costly, and we cannnot use our own custom fine-tuned model.

So, is it possible to use an open-source model running locally, such as Llama 3.1 ? The answer is yes, but you’ll need to overcome several challenges, as the official implementation doesn’t support local models, and you’ll need to modify some things manually.

You might wonder why not use LLMGraphTransformer from LangChain instead? I can only say that, as of now, it is still an experimental feature and differs slightly from Microsoft’s official implementation. If you’re interested, you can check out the following links:

• https://python.langchain.com/v0.2/docs/how_to/graph_constructing/
• https://python.langchain.com/v0.2/docs/tutorials/graph/

Prerequisites

• Install Ollama and have it running.
• An LLM model, such as ollama pull llama3.1:8b.
• A Text Embedding model, such as ollama pull nomic-embed-text.

Troubleshooting Process

When I first opened the Get Started page, I was excited to see how short it was and thought I could get it working quickly. Looking back, I was overly optimistic…

The first few steps were straightforward and did not cause issues:

pip install graphrag
mkdir -p ./ragtest/input
curl https://www.gutenberg.org/cache/epub/24022/pg24022.txt > ./ragtest/input/book.txt
python -m graphrag.index --init --root ./ragtest

Next, I needed to edit settings.yaml, and that’s where the problems began. How should I configure the model?

First, delete .env because we don’t need the API key.

Then, open settings.yaml and input the following:

View settings.yaml on GitHub Gist

The differences compared to the original settings.yaml are:

• llm
  • api_key: Set it to anything. It’s not important since we won’t be using an API. I set it to NONE.
  • model: Set this to your local LLM model, such as llama3.1:8b.
  • max_tokens: Set this to a number smaller than the maximum token limit for your LLM model.
  • api_base: Set this to http://localhost:11434/v1.
• embeddings
  • llm
    • api_key: Again, set it to anything since we won’t use the API. I set it to NONE.
  • model: Set this to your local Text Embedding model, such as nomic-embed-text.
  • api_base: Set this to http://localhost:11434/api.
  • batch_max_tokens: Set this to a number smaller than the maximum token limit for your Text Embedding model.
• chunks

  • size: Lower this to 300. If not adjusted, you’ll encounter the following error. More details in the related issue: https://github.com/microsoft/graphrag/issues/362

    09:44:07,997 datashaper.workflow.workflow ERROR Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key
    Traceback (most recent call last):
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
        result = node.verb.func(**verb_args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
        output_df[[level_to, to]] = pd.DataFrame(
        ~~~~~~~~~^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/frame.py", line 4299, in __setitem__
        self._setitem_array(key, value)
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
        check_key_length(self.columns, key, value)
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
        raise ValueError("Columns must be same length as key")
    ValueError: Columns must be same length as key
    16:25:46,215 graphrag.index.reporting.file_workflow_callbacks INFO Error executing verb "cluster_graph" in create_base_entity_graph: Columns must be same length as key details=None
    16:25:46,216 graphrag.index.run ERROR error running workflow create_base_entity_graph
    Traceback (most recent call last):
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/run.py", line 325, in run_pipeline
        result = await workflow.run(context, callbacks)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 369, in run
        timing = await self._execute_verb(node, context, callbacks)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 410, in _execute_verb
        result = node.verb.func(**verb_args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/graph/clustering/cluster_graph.py", line 102, in cluster_graph
        output_df[[level_to, to]] = pd.DataFrame(
        ~~~~~~~~~^^^^^^^^^^^^^^^^
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/frame.py", line 4299, in __setitem__
        self._setitem_array(key, value)
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/frame.py", line 4341, in _setitem_array
        check_key_length(self.columns, key, value)
      File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/pandas/core/indexers/utils.py", line 390, in check_key_length
        raise ValueError("Columns must be same length as key")
    ValueError: Columns must be same length as key
    

After changing the settings, excitedly run python -m graphrag.index --init --root ./ragtest, and you’ll get the following error:

09:19:54,508 datashaper.workflow.workflow ERROR Error executing verb "text_embed" in create_final_entities: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 415, in _execute_verb
    result = await result
             ^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/text_embed.py", line 105, in text_embed
    return await _text_embed_in_memory(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/text_embed.py", line 130, in _text_embed_in_memory
    result = await strategy_exec(texts, callbacks, cache, strategy_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 62, in run
    embeddings = await _execute(llm, text_batches, ticker, semaphore)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 106, in _execute
    results = await asyncio.gather(*futures)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 100, in embed
    chunk_embeddings = await llm(chunk)
                       ^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/caching_llm.py", line 96, in __call__
    result = await self._delegate(input, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 177, in __call__
    result, start = await execute_with_retry()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 159, in execute_with_retry
    async for attempt in retryer:
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 165, in execute_with_retry
    return await do_attempt(), start
           ^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 147, in do_attempt
    return await self._delegate(input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/base_llm.py", line 49, in __call__
    return await self._invoke(input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/base_llm.py", line 53, in _invoke
    output = await self._execute_llm(input, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/openai_embeddings_llm.py", line 36, in _execute_llm
    embedding = await self.client.embeddings.create(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/resources/embeddings.py", line 237, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1613, in _request
    return await self._process_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1710, in _process_response
    return await api_response.parse()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_response.py", line 420, in parse
    parsed = self._options.post_parser(parsed)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/resources/embeddings.py", line 225, in parser
    for embedding in obj.data:
TypeError: 'NoneType' object is not iterable
09:19:54,533 graphrag.index.reporting.file_workflow_callbacks INFO Error executing verb "text_embed" in create_final_entities: 'NoneType' object is not iterable details=None
09:19:54,541 graphrag.index.run ERROR error running workflow create_final_entities
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/run.py", line 325, in run_pipeline
    result = await workflow.run(context, callbacks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 369, in run
    timing = await self._execute_verb(node, context, callbacks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/datashaper/workflow/workflow.py", line 415, in _execute_verb
    result = await result
             ^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/text_embed.py", line 105, in text_embed
    return await _text_embed_in_memory(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/text_embed.py", line 130, in _text_embed_in_memory
    result = await strategy_exec(texts, callbacks, cache, strategy_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 62, in run
    embeddings = await _execute(llm, text_batches, ticker, semaphore)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 106, in _execute
    results = await asyncio.gather(*futures)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/index/verbs/text/embed/strategies/openai.py", line 100, in embed
    chunk_embeddings = await llm(chunk)
                       ^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/caching_llm.py", line 96, in __call__
    result = await self._delegate(input, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 177, in __call__
    result, start = await execute_with_retry()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 159, in execute_with_retry
    async for attempt in retryer:
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 166, in __anext__
    do = await self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/tenacity/__init__.py", line 398, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 165, in execute_with_retry
    return await do_attempt(), start
           ^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/rate_limiting_llm.py", line 147, in do_attempt
    return await self._delegate(input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/base_llm.py", line 49, in __call__
    return await self._invoke(input, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/base/base_llm.py", line 53, in _invoke
    output = await self._execute_llm(input, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/openai_embeddings_llm.py", line 36, in _execute_llm
    embedding = await self.client.embeddings.create(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/resources/embeddings.py", line 237, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1816, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1510, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1613, in _request
    return await self._process_response(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_base_client.py", line 1710, in _process_response
    return await api_response.parse()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/_response.py", line 420, in parse
    parsed = self._options.post_parser(parsed)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/openai/resources/embeddings.py", line 225, in parser
    for embedding in obj.data:
TypeError: 'NoneType' object is not iterable

The solution comes from this GitHub comment .

The comment suggests modifying a file’s content, but I don’t think that’s a good idea, as it might cause issues when using the package’s original functionality later. So, I opted for an external monkey patch approach. First, create a file called monkey_patch.py and input the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

def patch_openai_embeddings_llm():
    from graphrag.llm.openai.openai_embeddings_llm import OpenAIEmbeddingsLLM
    import ollama

    async def _execute_llm(self, input, **kwargs):
        embedding_list = []
        for inp in input:
            embedding = ollama.embeddings(model=self.configuration.model, prompt=inp)
            embedding_list.append(embedding["embedding"])
        return embedding_list

    OpenAIEmbeddingsLLM._execute_llm = _execute_llm

Make sure to install ollama by running pip install ollama.

We download the graphrag.index CLI content and monkey-patch it.

First, run

wget https://raw.githubusercontent.com/microsoft/graphrag/v0.3.2/graphrag/index/__main__.py -O index.py

Then, replace line 8 of index.py:

1

from .cli import index_cli

with the following:

1
2
3

from graphrag.index.cli import index_cli
from monkey_patch import patch_openai_embeddings_llm
patch_openai_embeddings_llm()

Now, run python index.py --root ./ragtest. Finally, no more errors!

Try executing a Global Query:

python -m graphrag.query \
--root ./ragtest \
--method global \
"What are the top themes in this story?"

This error occurs:

Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
not expected dict type. type=<class 'str'>:
Traceback (most recent call last):
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/llm/openai/utils.py", line 130, in try_parse_json_object
    result = json.loads(input)
             ^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Try executing a Local Query:

python -m graphrag.query \
--root ./ragtest \
--method local \
"Who is Scrooge, and what are his main relationships?"

This error occurs:

Error embedding chunk {'OpenAIEmbedding': "'NoneType' object is not iterable"}
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/__main__.py", line 92, in <module>
    run_local_search(
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/cli.py", line 164, in run_local_search
    response, context_data = asyncio.run(
                             ^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/api.py", line 222, in local_search
    result: SearchResult = await search_engine.asearch(query=query)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/structured_search/local_search/search.py", line 67, in asearch
    context_text, context_records = self.context_builder.build_context(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/structured_search/local_search/mixed_context.py", line 139, in build_context
    selected_entities = map_query_to_entities(
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/context_builder/entity_extraction.py", line 55, in map_query_to_entities
    search_results = text_embedding_vectorstore.similarity_search_by_text(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/vector_stores/lancedb.py", line 118, in similarity_search_by_text
    query_embedding = text_embedder(text)
                      ^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/context_builder/entity_extraction.py", line 57, in <lambda>
    text_embedder=lambda t: text_embedder.embed(t),
                            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/llm/oai/embedding.py", line 96, in embed
    chunk_embeddings = np.average(chunk_embeddings, axis=0, weights=chunk_lens)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/numpy/lib/function_base.py", line 550, in average
    raise ZeroDivisionError(
ZeroDivisionError: Weights sum to zero, can't be normalized

So, I had to monkey-patch the query CLI as well. Related issues and comments:

• https://github.com/microsoft/graphrag/issues/345#issuecomment-2230317752
• https://github.com/microsoft/graphrag/issues/575#issuecomment-2252045859

Add the following two functions to monkey_patch.py:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114

def patch_query_embedding():
    from graphrag.query.llm.oai.embedding import OpenAIEmbedding
    import ollama
    from tenacity import (
        AsyncRetrying,
        RetryError,
        Retrying,
        retry_if_exception_type,
        stop_after_attempt,
        wait_exponential_jitter,
    )

    def _embed_with_retry(self, text, **kwargs):
        try:
            retryer = Retrying(
                stop=stop_after_attempt(self.max_retries),
                wait=wait_exponential_jitter(max=10),
                reraise=True,
                retry=retry_if_exception_type(self.retry_error_types),
            )
            for attempt in retryer:
                with attempt:
                    embedding = (ollama.embeddings(model=self.model, prompt=text)["embedding"] or [])
                    return (embedding, len(text))
        except RetryError as e:
            self._reporter.error(
                message="Error at embed_with_retry()",
                details={self.__class__.__name__: str(e)},
            )
            return ([], 0)
        else:
            # TODO: why not just throw in this case?
            return ([], 0)

    async def _aembed_with_retry(self, text, **kwargs):
        try:
            retryer = AsyncRetrying(
                stop=stop_after_attempt(self.max_retries),
                wait=wait_exponential_jitter(max=10),
                reraise=True,
                retry=retry_if_exception_type(self.retry_error_types),
            )
            async for attempt in retryer:
                with attempt:
                    embedding = (ollama.embeddings(model=self.model, prompt=text)["embedding"] or [])
                    return (embedding, len(text))
        except RetryError as e:
            self._reporter.error(
                message="Error at embed_with_retry()",
                details={self.__class__.__name__: str(e)},
            )
            return ([], 0)
        else:
            # TODO: why not just throw in this case?
            return ([], 0)

    OpenAIEmbedding._embed_with_retry = _embed_with_retry
    OpenAIEmbedding._aembed_with_retry = _aembed_with_retry

def patch_global_search():
    from graphrag.query.structured_search.global_search.search import GlobalSearch
    import logging
    import time
    from graphrag.query.llm.text_utils import num_tokens
    from graphrag.query.structured_search.base import SearchResult
    log = logging.getLogger(__name__)

    async def _map_response_single_batch(self, context_data, query, **llm_kwargs):
        """Generate answer for a single chunk of community reports."""
        start_time = time.time()
        search_prompt = ""
        try:
            search_prompt = self.map_system_prompt.format(context_data=context_data)
            search_messages = [ {"role": "user", "content": search_prompt + "\n\n### USER QUESTION ### \n\n" + query} ]
            async with self.semaphore:
                search_response = await self.llm.agenerate(
                    messages=search_messages, streaming=False, **llm_kwargs
                )
                log.info("Map response: %s", search_response)
            try:
                # parse search response json
                processed_response = self.parse_search_response(search_response)
            except ValueError:
                # Clean up and retry parse
                try:
                    # parse search response json
                    processed_response = self.parse_search_response(search_response)
                except ValueError:
                    log.warning(
                        "Warning: Error parsing search response json - skipping this batch"
                    )
                    processed_response = []

            return SearchResult(
                response=processed_response,
                context_data=context_data,
                context_text=context_data,
                completion_time=time.time() - start_time,
                llm_calls=1,
                prompt_tokens=num_tokens(search_prompt, self.token_encoder),
            )

        except Exception:
            log.exception("Exception in _map_response_single_batch")
            return SearchResult(
                response=[{"answer": "", "score": 0}],
                context_data=context_data,
                context_text=context_data,
                completion_time=time.time() - start_time,
                llm_calls=1,
                prompt_tokens=num_tokens(search_prompt, self.token_encoder),
            )

    GlobalSearch._map_response_single_batch = _map_response_single_batch

Run

wget https://raw.githubusercontent.com/microsoft/graphrag/v0.3.2/graphrag/query/__main__.py -O query.py

Replace line 9 of query.py:

1

from .cli import run_global_search, run_local_search

with the following:

1
2
3
4

from graphrag.query.cli import run_global_search, run_local_search
from monkey_patch import patch_query_embedding, patch_global_search
patch_query_embedding()
patch_global_search()

Then, run the Global and Local Search queries. Local Search looks good, while Global Search still needs parameter adjustments. But at least there are no more errors! Hooray!

$ python query.py --root ./ragtest --method global "What are the top themes in this story?"

INFO: Reading settings from ragtest/settings.yaml
/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/indexer_adapters.py:71: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  entity_df["community"] = entity_df["community"].fillna(-1)
/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/indexer_adapters.py:72: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  entity_df["community"] = entity_df["community"].astype(int)
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'llama3.1:8b', 'max_tokens': 8191, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}

SUCCESS: Global Search Response:
You didn't provide a story. Please share the text, and I'll be happy to help you identify the top themes.

$ python query.py --root ./ragtest --method local "Who is Scrooge, and what are his main relationships?"

INFO: Reading settings from ragtest/settings.yaml

INFO: Vector Store Args: {}
/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/indexer_adapters.py:71: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  entity_df["community"] = entity_df["community"].fillna(-1)
/home/chishengliu/miniforge3/envs/graphrag/lib/python3.11/site-packages/graphrag/query/indexer_adapters.py:72: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  entity_df["community"] = entity_df["community"].astype(int)
creating llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_chat", 'model': 'llama3.1:8b', 'max_tokens': 8191, 'temperature': 0.0, 'top_p': 1.0, 'n': 1, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/v1', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': True, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}
creating embedding llm client with {'api_key': 'REDACTED,len=4', 'type': "openai_embedding", 'model': 'nomic-embed-text', 'max_tokens': 4000, 'temperature': 0, 'top_p': 1, 'n': 1, 'request_timeout': 180.0, 'api_base': 'http://localhost:11434/api', 'api_version': None, 'organization': None, 'proxy': None, 'cognitive_services_endpoint': None, 'deployment_name': None, 'model_supports_json': None, 'tokens_per_minute': 0, 'requests_per_minute': 0, 'max_retries': 10, 'max_retry_wait': 10.0, 'sleep_on_rate_limit_recommendation': True, 'concurrent_requests': 25}

SUCCESS: Local Search Response:
A classic character!

Ebenezer Scrooge is the main protagonist of Charles Dickens' novella "A Christmas Carol". He is a miserly and bitter old man who lives in London during the Victorian era. Scrooge is known for his extreme love of money, his disdain for charity and kindness, and his general grumpiness.

Scrooge's main relationships are:

1. **Jacob Marley**: The ghost of his deceased business partner, who appears to Scrooge on Christmas Eve to warn him about the consequences of his selfish ways.
2. **Bob Cratchit**: His underpaid and overworked clerk, who is struggling to provide for his large family during the holiday season. Scrooge's treatment of Bob is particularly cruel, as he pays him a meager salary and expects him to work long hours without complaint.
3. **Tiny Tim**: The youngest son of Bob Cratchit, who suffers from illness and poverty. Scrooge's heartlessness towards Tiny Tim serves as a catalyst for his transformation in the story.
4. **His nephew, Fred**: A kind and generous young man who invites Scrooge to join him for Christmas dinner, but is rebuffed by his miserly uncle.
5. **The Ghosts of Christmas Past, Present, and Yet to Come**: Three supernatural visitations that haunt Scrooge on Christmas Eve, forcing him to confront the consequences of his actions and the possibility of a better future.

Through these relationships, Dickens explores themes of redemption, kindness, and the importance of human connection in "A Christmas Carol".

Summary

All the code from this article is in this Gist . Here’s a quick summary of the steps:

• Follow the official Get Started guide and run python -m graphrag.index --init --root ./ragtest.
• Replace settings.yaml with the one in the Gist.
• Run

  wget https://raw.githubusercontent.com/microsoft/graphrag/v0.3.2/graphrag/index/__main__.py -O index.py
  

• Run

  wget https://raw.githubusercontent.com/microsoft/graphrag/v0.3.2/graphrag/query/__main__.py -O query.py
  

• Download the graphrag_monkey_patch.py from the Gist.
• Replace line 8 of index.py with:

  1 2 3

  from graphrag.index.cli import index_cli from graphrag_monkey_patch import patch_graphrag patch_graphrag()

• Replace line 9 of query.py with:

  1 2 3

  from graphrag.query.cli import run_global_search, run_local_search from graphrag_monkey_patch import patch_graphrag patch_graphrag()

• Whenever you need to use python -m graphrag.index, switch to using python index.py instead. Similarly, for python -m graphrag.query, switch to python query.py.

I assume that you have a basic understanding of Kubernetes and know how to use kubectl.

Pod, ReplicaSet, Service, and even Namespace are actually types of API Resources.

You can use kubectl api-resources to view all the API Resources currently available in your Kubernetes cluster.

$ kubectl api-resources
NAME                     SHORTNAMES   APIVERSION   NAMESPACED   KIND
bindings                              v1           true         Binding
componentstatuses        cs           v1           false        ComponentStatus
configmaps               cm           v1           true         ConfigMap
endpoints                ep           v1           true         Endpoints
events                   ev           v1           true         Event
limitranges              limits       v1           true         LimitRange
namespaces               ns           v1           false        Namespace
nodes                    no           v1           false        Node
persistentvolumeclaims   pvc          v1           true         PersistentVolumeClaim
persistentvolumes        pv           v1           false        PersistentVolume
pods                     po           v1           true         Pod
podtemplates                          v1           true         PodTemplate
replicationcontrollers   rc           v1           true         ReplicationController
resourcequotas           quota        v1           true         ResourceQuota
secrets                               v1           true         Secret
serviceaccounts          sa           v1           true         ServiceAccount
services                 svc          v1           true         Service
...

Kubernetes Custom Resources

Custom Resources, as the name suggests, refer to custom API Resources that we can create and install in Kubernetes to extend its functionality.

According to the official documentation, there are two ways to create Custom Resources, but the most common method is using Custom Resource Definitions (CRD), so we will focus on CRDs here.

In fact, CRD itself is a type of API Resource, and this can be clearly seen with the following command:

$ kubectl api-resources | grep -i custom
customresourcedefinitions   crd,crds   apiextensions.k8s.io/v1   false   CustomResourceDefinition

Think about how we usually create Resources; typically, we write a YAML file and then create it with kubectl apply. The steps to create a CRD are the same.

First, create a file called chishengliu-crd.yaml with the following content:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: chishenglius.chishengliu.com
spec:
  group: chishengliu.com
  scope: Namespaced
  names:
    plural: chishenglius
    singular: chishengliu
    kind: ChiShengLiu
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                mood:
                  type: string

Then run the following command:

$ kubectl apply -f chishengliu-crd.yaml
customresourcedefinition.apiextensions.k8s.io/chishenglius.chishengliu.com created

$ kubectl api-resources | grep -i chishengliu
chishenglius   chishengliu.com/v1   true   ChiShengLiu

You will notice that there is now a new Resource in the cluster called ChiShengLiu.

To create this kind of ChiShengLiu Resource, you also write a YAML file and then use kubectl apply. Create a file called happy-chishengliu.yaml with the following content:

1
2
3
4
5
6

apiVersion: chishengliu.com/v1
kind: ChiShengLiu
metadata:
  name: happy-chishengliu
spec:
  mood: "happy"

Then run the following command:

$ kubectl apply -f happy-chishengliu.yaml
chishengliu.chishengliu.com/happy-chishengliu created

$ kubectl describe chishengliu happy-chishengliu
Name:         happy-chishengliu
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  chishengliu.com/v1
Kind:         ChiShengLiu
Metadata:
  Creation Timestamp:  2024-08-28T17:37:57Z
  Generation:          1
  Resource Version:    1609
  UID:                 3cbc6964-2c7b-4e60-b113-4fa5f4dd0b3c
Spec:
  Mood:  happy
Events:  <none>

You can see that we have created a ChiShengLiu Resource named happy-chishengliu.

Kubernetes Operator

Having a Custom Resource alone is not very useful. You might notice that after creating this ChiShengLiu Resource, nothing changes in the Kubernetes cluster, unlike with built-in Resources.

Built-in Resources have functionality because there are corresponding Controllers running in the cluster. The Controller’s job is to monitor certain Resources in the cluster and adjust the cluster to the state declared in the Resource’s spec. For example, the Replication Controller monitors all ReplicaSets in the cluster, and when any ReplicaSet is created, updated, or deleted, it adjusts the cluster to the corresponding state, which means creating the correct number of Pods.

Therefore, if we want the ChiShengLiu Resource we created earlier to have an effect, we need to install a Controller for ChiShengLiu in the cluster. This Controller is not built-in and needs to be written by ourselves, so it’s called a Custom Controller, also known as a Kubernetes Operator. The official documentation refers to this pattern as the Operator Pattern.

The most common use case for Kubernetes Operators is to encapsulate underlying logic, especially for framework or tool developers, allowing users to install everything by simply applying a YAML file with kubectl apply, with the Operator handling the rest.

So, how do we write a Kubernetes Operator? Stay tuned for future updates in this series.

References

• Custom Resources | Kubernetes
• Operator pattern | Kubernetes

There are a few essential documents that you must read. These might be .md files, .rst files, or specific pages on the project’s website:

• README.md: The file that introduces the project.
• Contribution Guide: This outlines how to contribute to the project, including any formatting or styling rules you need to follow when submitting contributions.
• Code of Conduct: The community’s behavior guidelines. Generally, as long as you are not too rude or disrespectful, you should be fine, but it’s still important to read through it.

Issue Tracker

Understand what issue tracker the project uses. Common ones include GitHub issues or JIRA. As a beginner, it’s typical to start by looking for existing issues to work on. If you need to open an issue, be sure to search first to see if a similar issue has already been opened.

Commits

Check whether the project has specific requirements for commit message formatting. It’s a good habit not to write commit messages in a sloppy manner. If there are no special requirements, you can refer to Conventional Commits . The basic format looks like this:

<type>[optional scope]: <description>

[optional body]

[optional footer(s)]

Example:

feat: Add XXX feature.

fix(frontend): Fix XXX bugs.

The most common types are:

• feat: Add new features.
• fix: Bug fixes.
• docs: Documentation only changes.
• refactor: Refactoring. That means it does not change any existing functionalities.
• test: Add or correcting existing tests.
• chore: Miscellaneous tasks; use chore if you don’t know how to classify it.

Some projects might have slight variations in their symbols or casing, such as:

[Feat] Add XXX feature.

[Fix][Frontend] Fix XXX bugs.

Additionally, some projects provide pre-commit hooks. If so, follow the instructions to install them.

Some projects require every commit to be sign-offed. For details, please refer to the other tutorial written by me: The Ultimate Guide to Git for Open-Source Development .

Pull Request

Unless there are special circumstances, one Pull Request usually corresponds to one issue. Generally, before you submit a Pull Request, if there is no corresponding issue, you should open an issue first. However, if the project does not strictly enforce this rule, and your Pull Request contains only minor changes, you can also directly submit a Pull Request.

Some Special Terms

Sometimes reviewers leave comments using uncommon abbreviations or terms. Besides understanding them, you can also learn to use these terms, which will make you feel more professional 😂.

• LGTM: Looks good to me. Indicates that the reviewer thinks your code is fine.
• PTAL: Please take a look. A request for someone else to review.
• WIP: Work in progress. Indicates that this PR is not ready yet.
• ditto: Means “same as above”.
• nit: Indicates a suggestion for a change that the reviewer considers minor, so you can choose whether to make it or not. For example: nit: Rename this variable to XXX.

As an engineer, Git is an essential tool, and Lazygit is a command-line Git tool written in Golang that can save you a significant amount of time by reducing the need to manually type Git commands. Saving a few seconds each time can add up to hours saved over the course of a year.

Don’t tell me you type commands quickly (you’re just too lazy to learn new tools), or that you use something like alias gs='git status' to speed things up (you’re just afraid of learning a lot of shortcuts). Trust me, nothing can be faster than using this tool.

I won’t go into the installation steps here; please refer to the official documentation for that.

Interface

After installation, you can start Lazygit in a repository by executing lazygit. I set up an alias alias lg='lazygit' to help me launch it even faster.

The interface is divided into the left and right sides. The left side has five sections: Status, Files, Local branches, Commits, and Stash. Some sections also have tabs, such as the Files section, which has tabs for Worktrees and Submodules.

The right side has two areas: a detailed information preview and a list of the corresponding Git commands triggered by each shortcut.

Frequently Used Shortcuts

You can find the complete list of shortcuts in the official documentation .

Here are a few shortcuts I use the most:

• Global
  • h and l: Switch between the sections on the left side.
  • k and j: Move the cursor up and down.
  • /: Search for a string.
  • [ and ]: Switch between the tabs on the left side.
  • Ctrl + o: Copy the file name (in the Files section), branch name (in the Local branches section), or commit hash (in the commits section) to the clipboard.
  • q: Exit Lazygit.
• Left Side - Files Section
  • Space: Add or remove the selected file from the staging area.
  • a: Add or remove all files from the staging area.
  • Enter: Collapse or expand folders when used on a folder, or enter the file preview area on the right side when used on a file.
  • C: Open an external text editor to edit the commit message, then commit.
  • A: Amend the last commit.
  • d: Discard all changes to the selected file that have not been committed. Note that if the file is newly created, it will be deleted.
  • D: Discard all changes to all files that have not been committed. Use this when you’ve messed up and want to start over.
  • S: Open the stash options.
• Left Side - Local Branches Section
  • Space: Switch to the selected branch.
  • n: Checkout a new branch from the selected branch.
  • f: Fetch new commits for the selected branch from the remote.
  • r: Rebase onto the selected branch.
  • R: Rename the selected branch.
  • p: Pull the selected branch.
  • P: Push the selected branch.
  • o: Open the GitHub pull request page for the selected branch.
  • Enter: View the commits of the selected branch. Use Esc to go back.
• Left Side - Commits Section
  • Enter: View the files changed in the selected commit. Use Esc to go back.
  • R: Open an external editor to modify the commit message.
  • d: Delete the selected commit.
  • g: Open the reset options.
  • F: Create a fixup! commit for the selected commit.
  • S: Squash all fixup! commits on top of the selected commit.
• Left Side - Stash Section
  • g: Apply and delete the selected stash.
  • Space: Apply the selected stash without deleting it.
  • d: Delete the selected stash.
• Right Side - Preview Area
  • v: Enter multi-line selection mode.
  • Space: Add or remove the selected line or lines (when in multi-line selection mode) from the staging area.
  • Esc: Return to the left side sections.

Practical Example

Let’s combine this with the previous article on The Ultimate Guide to Git for Open-Source Development and show how I use Lazygit to develop open-source projects.

Creating a New Branch

First, make sure that the upstream-master branch is in sync with the remote. Use h and l to switch to the Local branches section, then use k and j to navigate to the upstream-master branch. Next, press f to fetch the latest changes. Finally, press n to create a new branch.

Syncing with Upstream

Again, navigate to the upstream-master branch in the Local branches section, then press f to fetch the latest changes and r to rebase onto it.

Development and Pull Request

Navigate to the Files section and press Space to add files to the staging area one by one, or use a to add all files at once. Then, press C to commit. Next, navigate to the branch you want to push in the Local branches section and press P to push. Finally, press o to open the GitHub pull request page.

Conclusion

At first, you might find it difficult to learn all the shortcuts, but with time, they will become muscle memory. The time saved is definitely worth it.

If you want to learn more about the features, please visit the official repository.

To develop an open-source project, we must first Fork the Repository , as it is not our own project, and we don’t have write access.

Once we fork the repository, an identical copy will appear under our account. We usually refer to the original project as the upstream repo and the forked copy as the downstream repo.

Since we have write access to the forked repository, the first step is to clone it . Note that we should clone our forked repo, not the upstream repo.

For example, if you want to contribute to the kuberay project, and our GitHub username is MortalHappiness, the URL of the upstream repo would be https://github.com/ray-project/kuberay.git, and the URL of our forked repo should be git@github.com:MortalHappiness/kuberay.git (SSH) or https://github.com/MortalHappiness/kuberay.git (HTTPS). Let’s assume we are using SSH.

After cloning, we can run git remote -v, and it should look something like this:

origin  git@github.com:MortalHappiness/kuberay.git (fetch)
origin  git@github.com:MortalHappiness/kuberay.git (push)

Next, we need to add the upstream remote by executing the following commands:

1
2

git remote add upstream https://github.com/ray-project/kuberay.git
git remote set-url --push upstream no_push

In the above commands, replace the URL in the first line with the upstream URL of the project you want to contribute to. The second line is to prevent us from accidentally pushing to the upstream repo. (Although we don’t have write access and can’t push, if one day we become a committer with write access, this line helps avoid accidentally pushing to the upstream.)

Run git remote -v again, and it should look like this:

origin  git@github.com:MortalHappiness/kuberay.git (fetch)
origin  git@github.com:MortalHappiness/kuberay.git (push)
upstream        https://github.com/ray-project/kuberay.git (fetch)
upstream        no_push (push)

The following is my personal habit and is not necessarily required. To avoid accidentally developing features on the master branch (or main branch), I create a local branch to track the upstream/master branch and then delete the local master branch.

1
2
3

git fetch upstream
git checkout -b upstream-master upstream/master
git branch -d master

If you’re using the GitHub CLI , you can also set the default repo to upstream.

1

gh repo set-default https://github.com/ray-project/kuberay.git

Remember to replace the URL in the above command with your upstream URL.

Developing New Features

1. Ensure that our local upstream-master branch is in sync with the remote:

   1 2	git checkout upstream-master git pull upstream master

2. Check out a new branch, let’s say feature/example:

   1	git checkout -b feature/example upstream-master

3. Proceed with normal development (add, commit, push).

4. Go to the upstream repo to open a pull request .

Syncing Upstream

If we are in the middle of development and the upstream repo has new commits, we need to sync with the upstream by running git pull upstream master.

Sign-off Commits (DCO)

Some open-source projects require every commit to be signed off. If it’s not done, one of the CI check called DCO (Developer Certificate of Origin), will fail. The sign-off process is straightforward: when you run git commit, you add the -s flag, turning it into git commit -s. This flag automatically appends a Sign-off-by line at the end of your commit message. So, the commit message will look like this:

Some commit message.

Signed-off-by: Chi-Sheng Liu <chishengliu@chishengliu.com>

Note that this sign-off is different from GPG-signed commits .

If you forget to add the -s flag when committing, you can follow this guide to remedy it.

If you find it cumbersome to add the -s flag every time, you can use a commit-msg hook:

View this GitHub Gist

Write the content above into .git/hooks/commit-msg and then execute chmod +x .git/hooks/commit-msg to make it executable.

The Apache Software Foundation (ASF) is currently one of the world’s largest open-source software organizations, composed of developers from around the globe. The software under its umbrella follows the Apache License , and it has fostered many well-known projects such as Hadoop, Spark, Kafka, and Flink.

Roles in ASF Projects

According to the official website , each Apache project typically has the following roles:

• User: Uses the project’s software but does not participate in development.
• Developer (Contributor): Participates in project development but does not have write access to the repository.
• Committer: After making significant contributions, the PMCs may nominate you as a Committer, granting write access to the repository. If you don’t already have an Apache email, you’ll get one. For example, my email is chishengliu@apache.org .
• PMC Member: After more prolonged involvement, you may be nominated as a PMC Member, having the power to decide the project’s long-term direction, nominate active contributors as Committers, and vote on formal releases.
• PMC Chair: The PMC Chair is the leader among PMC Members, responsible for communicating with the board and additional duties.
• ASF Member: After becoming a PMC Member for multiple projects, you may be nominated as an ASF Member, with the right to elect and be elected to the board and incubate new projects.

What is Apache CommunityOverCode?

Apache CommunityOverCode, formerly known as ApacheCon, is an annual open-source software conference organized by the Apache Software Foundation, held in Asia, Europe, and North America. It was renamed to emphasize that “ a healthy community is a higher priority than good code .”

Why Attend Apache CommunityOverCode In-Person Instead of Online?

There are several obvious reasons for attending in person:

• Networking: Attending in person allows you to make more friends, meet speakers, build connections, and aid in career development.
• Atmosphere: Experience the vibe of an international software conference firsthand.
• Opportunity to Ask Questions: You have the chance to ask questions and discuss with speakers in person, which is not possible with online participation.
• Sightseeing: Attend the conference while also visiting local tourist attractions.
• Meet Online Friends: Meet in person those you have collaborated with on open-source projects but have never met due to geographical reasons.

How to Attend Apache CommunityOverCode In-Person?

Basically, there are three ways: buying a ticket, registering as a speaker, or applying to be a volunteer. Below is the registration page.

• Buying a ticket: Regular registration requires a fee. If you are a Committer (with an Apache email), you can get a discount. Additionally, students and unemployed individuals can apply for free tickets.
• Registering as a speaker: If accepted, you can get a free speaker ticket. Speakers can also give complimentary tickets to friends and family, allowing them to attend for free.
• Applying to be a volunteer: The Apache Travel Assistance Committee (TAC) is a committee that helps those who want to attend Apache events but are financially constrained. You can apply to be a volunteer on their website. Note that you don’t need to be a Committer to apply! Committers can log in directly with their Apache email, while non-Committers can register for an account. Based on the experience from some friends of mine, Gmail sometimes fails to receive the email, while Outlook is more reliable. If you don’t receive an email, try another email service. Once approved, TAC will cover your airfare, accommodation, and conference tickets, saving a lot of expenses.

My Thoughts on Some Keynotes

The sessions were divided into two parts: Keynotes and Guest Speakers. Keynotes usually cover broad topics aligning with the conference theme and arouse general interest. Guest Speakers are community members who submit topics for in-depth introduction and discussion of specific projects or fields. Keynotes are held in the main conference room in the morning, while Guest Speakers are spread across various rooms in the afternoon, allowing attendees to choose topics of interest.

Keynotes usually feature one or two speakers discussing a topic, but there are also unique formats like roundtable discussions with 4-5 people chatting on stage about a topic, and Lightning Talks where each person speaks on a topic for 5 minutes.

Below is the schedule for the first day of the conference, to give you a sense of the event. To avoid taking up too much space, the schedules for the second and third days are not included in detail.

Opportunities and Challenges of Globalization - Xu Wang, Junping Du, William Guo, Zili Chen

This roundtable discussion addressed many issues related to open-source and globalization, which I found particularly interesting.

Language Contradictions in Localizing and Globalizing Open-Source Communities

One point discussed was the language issue in open-source communities. To internationalize, English documentation is necessary; otherwise, contributors would be limited to local sources. However, if the target market is local, local language documentation is needed to compete with other products that have local language documentation. Maintaining multi-language documentation is challenging because not all developers are proficient in multiple languages. When a passage needs modification, multiple documents need updates, leading to inconsistent information. Therefore, if internationalization is required, it’s usually best to maintain a single English document.

Regional Preferences Hindering Project Promotion

Another issue is Product Market Fit (PMF). A project popular in one region may struggle to gain traction in another due to different user habits. One speaker mentioned a project using drag-and-drop no-code workflows that was popular in Asia but struggled in Western markets where users preferred coding workflows, making promotion difficult.

Cultural Differences Affecting Development Atmosphere and Values

Another interesting topic was the development atmosphere in different regions. A speaker noted that Western developers are accustomed to communication via email with responses taking over a day, while Chinese developers expect replies within minutes due to a more competitive atmosphere. This difference may require using different communication tools for various markets, increasing the maintenance burden. One speaker noted that short-term slowness does not mean long-term slowness, as it allows developers to carefully review code quality, reducing future problems. Furthermore, doing things slowly does not mean overall progress will be slow, as many features can be developed simultaneously by many companies, leading to rapid overall progress.

Open Source: How We Got Here, Where We’re Going - Bruce Perens

This speaker is the primary author of the Open Source Definition, which is now case law, with a US court enforcing that software should not be called Open Source if it doesn’t meet the definition. He is also the author of BusyBox and the second Debian project leader.

History and Current Challenges of Open Source

The speaker first discussed the history of open source and then addressed current challenges. Many major companies use various open-source projects without paying the maintainers, who often work voluntarily. When maintainers cannot handle all tasks alone, they may grant permissions to other contributors, some of whom might have malicious intentions, leading to disasters like the xz backdoor incident . Another challenge is the abundance of open-source licenses, requiring companies to spend money ensuring compliance.

“Post-Open”: Ending Free Use by Major Companies

The speaker proposed a concept called “ Post-Open ,” where open-source software remains free for individuals and small companies, but companies with annual revenues exceeding five million dollars must pay the open-source community. The funds would be distributed based on developers’ contributions and the number of companies using the software.

My Thoughts on “Post-Open”

I believe the concept is a wonderful vision, motivating open-source project maintainers who cannot rely solely on voluntary work. Passion will eventually be exhausted, and almost everyone has experienced burnout before. I have immense respect for contributors to open-source projects. Although fulfilling this initiative will face many challenges, I believe the speaker can achieve this vision, just as he made open source a benchmark for U.S. courts.

Too Many Talks to Choose From? How to Select the Right Topic for You?

It may be hard to choose which talk to attend if this is your first time at such a conference, especially when you see so many guest speakers. I recommend you choose:

1. Topics you are familiar with. For example, “The best practice of integration technology driven by event in cloud - Xiaohui Wu, Alan Liu” by RedHat engineers on Kubernetes and microservices was easy for me to understand due to my experience writing Kubernetes Operators in open-source projects.
2. Non-technical talks like “Developing Soft Skills for a successful Open Source career - Sumaiya Nalukwago.” Non-technical topics do not require base knowledge, so almost all of them can be easily understood.
3. Popular technical topics like “Navigating the Lakehouse with Confidence: Best Practices for Implementation with Apache - Bill Zhang” by Cloudera, discussing Apache Iceberg , can also be chosen to learn about the applications and trends of these popular topics.

Volunteer Experience

Welcome Dinner

On the day before the event started, the Travel Assistance Committee invited volunteers to a restaurant called “乾塘大院” for a welcome dinner. The evening was filled with lively conversations and a friendly atmosphere as volunteers from various backgrounds came together. It was interesting to meet people from different countries and learn about each other’s cultures, sharing stories and experiences. The dinner gave us a chance to make new friends and feel like part of a team. We talked about our work in open-source projects and shared our excitement for the upcoming conference. The evening helped us understand different cultures better and set a positive tone for the days ahead.

Volunteer Tasks

Volunteering was not very demanding. We only needed to sit at the Apache booth for a 2-hour shift over the 3 days. There were many stickers of different projects on the table, and we had to inform passersby that they could take the stickers freely. Initially, the stickers were scattered in a pile, but because I found it a bit boring to just sit there, I started organizing them.

Additionally, we were supposed to help at the registration desk for a day, but since there were already local staff members there, I didn’t have much to do.

Lastly, we needed to count the number of attendees at Guest Speaker sessions and report it to the conference to help them understand which topics were most attended. This was also an easy task.

Certificate

After the event, volunteers received a beautifully framed certificate. It was a nice gesture!

Conclusion

Participating in the Apache CommunityOverCode Asia 2024 was an incredible experience, and I highly recommend that everyone attend at least once!

Can you believe it? Except for the domain name, the setup of my personal website is completely free! After all, this is just a site where I plan to post casual articles, so I’ll save money wherever I can. Let me explain how I built this website.

Framework Selection

Since I wanted it to be free, I couldn’t choose an SSR (Server-Side Rendering) framework like WordPress, as deploying it to a host would cost money. Although some cloud platforms offer free tiers, increased traffic would inevitably incur costs. Therefore, choosing an SSG (Static Site Generation) framework and deploying it on a platform offering free static site hosting is more sensible.

My previous website used the Gatsby framework, for a simple reason: I am very familiar with React , so I thought customization would be easier. However, I later realized I didn’t have time to maintain a bunch of Typescript and Javascript, which made me reluctant to update my website. This time, I choose Hugo . Maintaining a small amount of Go Template is easier, and I am familiar with Golang, too.

Theme Selection

Hugo’s official website offers many themes to choose from. This time, I chose the hugo-PaperMod theme. Themes usually provide download steps, just follow them.

1
2
3
4

hugo new site personal-website --format yaml
cd personal-website
git submodule add --depth=1 https://github.com/adityatelange/hugo-PaperMod.git themes/PaperMod
git submodule update --init --recursive

Configuration

You’ll have to follow the documentation provided by different themes to configure them to your liking. So in the following I only discuss some challenging configurations.

Multilingual

The theme I chose supports multiple languages, but some parts are not well translated. You’ll need to look at the source code and find a solution, or search the issues to see if anyone has provided solutions. Luckily, I found some solutions:

• https://github.com/adityatelange/hugo-PaperMod/issues/46

Last Modified Time

Similarly, search the issues for solutions:

• https://github.com/adityatelange/hugo-PaperMod/issues/1037

Chinese Fonts

Themes usually don’t specifically set Chinese fonts, so Chinese characters look ugly. I often use web fonts to solve this, which is more convenient. Here, I use Google’s Noto Sans Traditional Chinese font .

First, copy the font’s embed code by clicking “Get fonts”

Then click “Get embed code”

Then click “copy code”

Next, find where to insert HTML tags in the theme’s source code. After searching, I found this file . Create a file called layouts/partials/extend_head.html and write the copied content into it.

Then, find where the CSS controlling the fonts is located. After searching, I found this line: https://github.com/adityatelange/hugo-PaperMod/blob/9ea3bb0e1f3aa06ed7715e73b5fabb36323f7267/assets/css/core/reset.css#L27 .

Create a file called assets/css/extended/custom-font.css and write the following content:

1
2
3
4

body:lang(zh-tw) {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Oxygen,
    Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", "Noto Sans TC", sans-serif;
}

Note that this part is directly copied from the original code, adding the selector body:lang(zh-tw) to override the font style for Traditional Chinese. Since my articles mix Chinese and English, to avoid changing the English font, “Noto Sans TC” should be placed before the fallback font, which is before sans-serif.

Comment System

Since our site is a static site, we need to rely on external services to provide a comment system. The most famous one is Disqus . However, its free plan forces use to show a lot of ADs to the users, which I don’t like. So I chose giscus , which uses GitHub Discussions as the storage place for comments. The only downside is that it doesn’t allow users to log in and comment with Google or social media accounts like Disqus does, but since my articles are mostly technical, people reading them are likely to have a GitHub account, so it’s not a big issue.

It’s a bit tricky to support multilingual and light/dark mode switching for this comment system, so I had to write some Javascript. Here’s what it roughly looks like (sensitive information has been replaced):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

<div id="giscus-script" />

<script>
  let lang = document.documentElement.lang;
  let category = "English Comments";
  let categoryId = "<your-category-id>";
  if (lang === "zh-tw") {
    lang = "zh-TW";
    category = "Traditional Chinese Comments";
    categoryId = "<your-category-id>";
  }

  let theme = localStorage.getItem("pref-theme");
  if (theme !== "light" && theme !== "dark") {
    theme = window.matchMedia('(prefers-color-scheme: dark)').matches ? "dark" : "light";
  }

  const giscusAttributes = {
    "src": "https://giscus.app/client.js",
    "data-repo": "<your-repo>",
    "data-repo-id": "<your-repo-id>",
    "data-category": category,
    "data-category-id": categoryId,
    "data-mapping": "pathname",
    "data-strict": "1",
    "data-reactions-enabled": "1",
    "data-emit-metadata": "0",
    "data-input-position": "top",
    "data-theme": theme,
    "data-lang": lang,
    "data-loading": "lazy",
    "crossorigin": "anonymous",
    "async": "",
  };
  const giscusScript = document.createElement("script");
  Object.entries(giscusAttributes).forEach(([key, value]) =>
    giscusScript.setAttribute(key, value)
  );
  document.getElementById("giscus-script").appendChild(giscusScript);

  function setGiscusTheme(theme) {
    if (giscusScript) {
      giscusScript.remove();
      const newGiscusScript = document.createElement("script");
      Object.entries(giscusAttributes).forEach(([key, value]) =>
        newGiscusScript.setAttribute(key, value)
      );
      newGiscusScript.setAttribute('data-theme', theme); // Set the new theme
      document.getElementById("giscus-script").appendChild(newGiscusScript);
    }
  }

  function handleStorageChange() {
    const theme = localStorage.getItem('pref-theme');
    setGiscusTheme(theme);
  }

  window.addEventListener("storage", (event) => {
    if (event.key === 'pref-theme') {
      handleStorageChange();
    }
  });

  // Override the localStorage setItem method to detect changes in the same browsing context
  const originalSetItem = localStorage.setItem;
  localStorage.setItem = function(key, value) {
    originalSetItem.apply(this, arguments);
    if (key === 'pref-theme') {
      handleStorageChange();
    }
  };
</script>

Deployment

There are many platforms for Static Site Hosting, including GitHub Pages , Netlify , Cloudflare Pages , etc. I chose Cloudflare Pages for a simple reason: its free plan has the fewest restrictions. It even has no bandwidth limit!

Deployment is straightforward. Follow the official documentation, connect to the GitHub repo, and choose hugo as the build command.

CMS

It’s inconvenient to have to sit in front of a computer and open a text editor to write a blog post every time. A common solution is to use a CMS (Content Management System). I used to use Decap CMS (formally Netlify CMS), but it wasn’t very convenient for connecting to a GitHub repo, and it didn’t generate a blank line between YAML frontmatter and the body, which sometimes caused issues. So this time, I used Tina CMS . Fortunately, Tina Cloud is free for up to two users.

Setting up Tina CMS is not difficult. Follow the official setup instructions, then go to Cloudflare Pages and change the build command to git fetch --unshallow && npx --yes tinacms build && [ "$CF_PAGES_BRANCH" = "main" ] && hugo --minify || hugo -b $CF_PAGES_URL --minify and add the environment variables. It will look like this:

SEO

Favicon

A Favicon is a small icon on a webpage. Without setting it, it appears as a globe in Chrome, which doesn’t look good.

Go to Favicon Generator , upload an image, download the files, unzip them, and copy all the files into the static folder. Then refer to the theme documentation to see how to set the favicon.

OpenGraph

OpenGraph is used to set images and descriptions when your website is shared on social media. You can use https://www.opengraph.xyz/ to check how your website will look when shared on social media.

Google Search Console

Submitting your site to Google Search Console can speed up indexing. Remember to submit the sitemaps (both XML and RSS sitemaps) to speed up indexing furthermore.

LightHouse

LightHouse can be used to measure website performance and SEO. First, open the website in incognito mode to avoid interference from Chrome extensions. Then press Ctrl+Shift+J to enter Chrome Dev Tools, and select the LightHouse tab.

Select Desktop and click “Analyze page load”.

Results:

Note: Google Web Font Performance Tuning

When using Lighthouse, you might notice a performance issue related to Google Web Fonts. In this case, you can use the preload method to solve it:

1	<link href="https://fonts.googleapis.com/css2?family=Noto+Sans+TC:wght@100..900&display=swap" rel="preload" as="font" crossorigin="anonymous">

Page Analytics

To analyze website data, such as the number of visitors, the most commonly used tool is Google Analytics , which is beyond the scope of this article, so it is not detailed here.

Conclusion

Deploying a website requires attention to many details. This article only briefly covers many topics, as different frameworks and themes require different configurations. Although you can’t directly replicate a website by reading this article, I hope it helps you notice these details when deploying your personal website.

A few days ago, while developing the KubeRay project, I learned about a Kubernetes behavior from the issue’s comment section . There are two types of Eviction: Node-pressure Eviction and API-initiated Eviction . API-initiated Eviction is done by directly calling the API or using commands like kubectl drain. Pods evicted this way will ultimately be deleted and usually recreated on another node. However, for Node-pressure Eviction, kubelet will only set the Pod’s Phase to Failed without deleting it. Therefore, if the controller does not handle it properly, the Pod will not be recreated on another node.

Here’s a brief overview of the issue: When a Pod created by the KubeRay operator is on a node with insufficient disk space, the Pod gets evicted. After the disk space is cleared, the Pod remains in a Failed state and is not recreated on another node.

Now, I need to reproduce this issue. The key point is that since these two types of Evictions behave differently, I cannot use kubectl drain or similar commands to reproduce the scenario. I need to specifically create a Node-pressure Eviction. However, I don’t have a cluster to use; I do all my development on my personal computer, making it difficult to reproduce the issue. When developing Kubernetes applications locally, most people use minikube , kind , or k3d . Since I need a multi-node environment, minikube is excluded. Although it now supports multiple nodes , it’s still more commonly used for single-node scenarios. Both kind and k3d use Docker containers as Kubernetes nodes. My operating system is Linux Mint, and Docker runs natively, unlike macOS where Docker runs in a virtual machine. Because the resources (memory, disk, etc.) are shared between Docker and my local machine, if I do create a Node-pressure scenario, my computer might become unusable.

After extensive Googling, I discovered that Docker can set Runtime Memory Limits , and k3d has a --agents-memory flag to set agent node memory . This is how I found a way to reproduce the issue .

Steps

First, create a k3d cluster with 2 agent nodes, each with 3GB of memory, and trigger Pod Eviction when the available memory is less than 1GiB.

1
2
3
4
5
6

k3d cluster create \
  --agents 2 \
  --k3s - arg "--disable=traefik@server:0" \
  --agents - memory 3g \
  --k3s - arg "--kubelet-arg=eviction-hard=memory.available<1Gi@agent:0" \
  --k3s - arg "--kubelet-arg=eviction-hard=memory.available<1Gi@agent:1"

Check the memory of all nodes

1

kubectl get nodes -o custom-columns=NAME:.metadata.name,CAPACITY_MEMORY:.status.capacity.memory,ALLOCATABLE_MEMORY:.status.allocatable.memory

Output:

# NAME                       CAPACITY_MEMORY   ALLOCATABLE_MEMORY
# k3d-k3s-default-agent-1    3221225Ki         2172649Ki
# k3d-k3s-default-agent-0    3221225Ki         2172649Ki
# k3d-k3s-default-server-0   32590664Ki        32590664Ki

You can see that both agent 0 and agent 1 have 3GB of memory, but only 2GB is allocatable because Pod Eviction is triggered when available memory is less than 1GiB.

Next, add taints to agent 0 and agent 1 so that subsequent Pods will only be deployed to the server-0 node.

1
2

kubectl taint nodes k3d-k3s-default-agent-0 k3d=noschedule:NoSchedule
kubectl taint nodes k3d-k3s-default-agent-1 k3d=noschedule:NoSchedule

Install the KubeRay operator, so the operator Pod will run on the server-0 node.

1

helm install kuberay-operator kuberay/kuberay-operator --namespace ray-system --version 1.1.1 --create-namespace

Remove the taints from agent 0 and agent 1 and add a taint to server 0 so that subsequent Pods will not be deployed to server 0.

1
2
3

kubectl taint nodes k3d-k3s-default-server-0 k3d=noschedule:NoSchedule
kubectl taint nodes k3d-k3s-default-agent-0 k3d=noschedule:NoSchedule-
kubectl taint nodes k3d-k3s-default-agent-1 k3d=noschedule:NoSchedule-

Install the RayCluster custom resource. After installation, the KubeRay operator will create a head pod and a worker pod. Since the memory resource request for the head pod is 2GB and for the worker pod is 1GB in the helm chart, and both agent 0 and agent 1 have only 2GB of allocatable memory, these two Pods will definitely not be on the same node.

1

helm install raycluster kuberay/ray-cluster --version 1.1.1

Next, we need to perform a memory stress test on the node where the head pod is located. After some Googling, I found that stress-ng is commonly used for this purpose, so I’ll use it as well. We need to ensure that the head pod has stress-ng available. The simplest way is to copy the statically compiled stress-ng binary directly into the head pod, so we don’t have to worry about the head pod’s base image or any missing dependencies. As for obtaining the statically compiled binary, you can compile it yourself, but I took a shortcut by copying it from a Docker image that includes the binary . Assuming the head pod is named raycluster-kuberay-head-ldg9f.

1

kubectl cp ./stress-ng raycluster-kuberay-head-ldg9f:/home/ray

Open a shell on the head pod

1

kubectl exec -it raycluster-kuberay-head-ldg9f -- bash

Simulate memory stress

1

./stress-ng --vm 4 --vm-bytes 2G --vm-keep

In this way, you can see the head pod being evicted due to Node-pressure Eviction.