GenAI - CS5483

Generative AI (GenAI) is a powerful tool in programming. You can play with them in various ways in the Jupyter server as explained in this notebook.

Integration with Jupyter¶

GenAI is integrated into the Jupyter interface using jupyter-ai. You can interact with it directly inside the notebook or through a chat panel.

Jupyter Notebook¶

To be able to chat with a large language model (LLM) directly inside a notebook, run the line magic below:

if not input('Load JupyterAI? [Y/n]').lower()=='n':
    %reload_ext jupyter_ai

Load JupyterAI? [Y/n]

After loading the jupyter_ai extension, run the following cell magic to ask an LLM what an LLM is.

%%ai
Explain what LLM is in one line.

Let’s try different output formats:

%%ai -f math
What is the Pythagoras theorem?

%%ai -f html
What is the Pythagoras theorem?

%%ai -f html
Illustrate the Pythagoras theorem using a diagram in SVG.

%%ai
Can I trust you?

%%ai
Can you give me answers to assignments directly?

Regardless of what the response is, LLM is surprisingly good at many areas, especially programming at a basic level, thanks to the corpus of good programs and software documentations. Let’s use the following exercise as an example:

def gcd(a, b):
    ### BEGIN SOLUTION
    return gcd(b%a, a) if a else abs(b)
    ### END SOLUTION

# tests
assert gcd(48, 18) == 6
assert gcd(-20, 30) == 10

# hidden tests
# DON'T remove this cell
### BEGIN HIDDEN TESTS
assert gcd(-55, 89) == 1
### END HIDDEN TESTS

Don’t worry! You are allowed to use GenAI for formative assessments (such as lab assignments) but not summative assessments (midterm quiz and final examination). So, let’s use it now:

%%ai
Answer Exercise 1.

Garbage-in, Garbage out. We should give clear, context‑rich prompts to sharpen the model’s focus to give more relevant responses.

%%ai
Write a function `gcd` to calculate the GCD of two non-zero integers.

Mission accomplished? Two key questions to reflect on this approach to GenAI:

Is the program efficient?
If it is, have I gained enough understanding to replicate the solution for similar problems?

As an example, the following prints a one-liner solution for the GCD.

def gcd(a, b): return gcd(b%a, a) if a else abs(b)  # one-liner for GCD
gcd_code = ''.join(In[-1].split('\n')[:-2])   # storing the code to query AI
print(gcd_code)

def gcd(a, b): return gcd(b%a, a) if a else abs(b)  # one-liner for GCD

What’s worth learning are the ingredients involved in writing such program:

%%ai
Explain the ingredients involved in writing the following program:
--
{gcd_code}

What is challenging is the computational thinking skill involved that you will develop over time, mostly outside the classroom. Dive into it with the help of AI:

%%ai
Explain concisely in a paragraph or two how Euclidean algorithm works and 
whether there are better methods.

Chat Panel¶

There is also a chat interface to chat with an LLM called Jupyternaut:

Click the chat icon on the left menu bar. A chat panel will open.

Enter some prompts to chat with Jupyternaut:

```
What is Computational Thinking?
```

What is Data Mining and Data Warehousing?

```
/clear
```

You can add a cell’s text as context for Jupyternaut. Try this on the following code:^[1]

Select the cell you want to use.
Switch to Edit mode if it is a markdown cell. (Double click or press Enter.)
Highlight the desired text inside the cell.
In the Jupyternaut panel, enter a prompt, click the down arrow button and choose Send message with selection.

from math import gcd   # yet another one-liner for gcd!

Another way to add context is through an embedding model:

/learn cs5483_25b/Tutorial1/ – tell the LLM to ingest the documents in Tutorial1.
/ask Summarize what I need to do for Tutorial1 – request a concise summary of the lab tasks.
/learn -d – delete everything that was previously learned.

%%ai
Explain how a chat model and an embedding model work together to give 
domain-specific response.

You may check the default model configuration as follows:

Click the gear icon (⚙️) at the top‑right of the chat panel.
In the Completion model dropdown, select DIVE :: chat.
In the Embedding model dropdown, select DIVE :: embed.
Click Save Changes.
Click the back arrow to return to the chat panel.
Send a test message to confirm the chat model works.

If you are using other AI services (e.g., Azure OpenAI, you may reconfigure JupyterAI to use them.

If you want to reset the configuration in case of error, remove the configuration folder using the shell capture below:

if input('Reset Jupyter AI configuration? [y/N]').lower() == 'y':
    !rm -rf ~/.local/share/jupyter/jupyter_ai

Reset Jupyter AI configuration? [y/N]

After deleting the folder, simply restart the Jupyter server to recreate it.

Integration with VSCode¶

GenAI has been integrated into the Visual Studio Code (VSCode) interface^[2] via the extensions below.

Continue: Configured to run our deployed models.

Continue¶

To open the current folder Lab0 in VSCode, execute the following cell to generate the link:

from vscode_link_generator import vscode

vscode('.')

Start a chat with
```
Explain @vscode_link_generator.py
```
Switch to agent, open the file vscode_link_generator.py, and ask:
```
Which file do I have open?
```

The configuration files for the Continue extension are stored in .continue under your home directory:

!ls ~/.continue

config.ts    dev_data  package.json  tsconfig.json
config.yaml  index     sessions      types

Similar to JupyterAI, you may reconfigure Continue to use other providers.

%%ai
Explain concisely how an AI agent is different from an AI chatbot.

If you want to reset the configuration in case of error, remove the configuration folder and then restart the Jupyter server to recreate it:

if input('Reset Jupyter AI configuration? [y/N]').lower() == 'y':
    !rm -rf ~/.continue

Reset Jupyter AI configuration? [y/N]

Glossary¶

line magic: A command in Jupyter notebook that starts with % such as %load_ext is called a line magic. It can be used to performs tasks to change the notebook behavior, such as loading the %%ai cell magic.
cell magic: A command in Jupyter notebook that starts with %% such as %%ai is called a cell magic. It can be used to change the beavior of the cell, such as sending the text as a prompt to an LLM.
shell capture: A command in Jupyter notebook that starts with ! or !! is called a shell capture. It can be used to execute a shell/terminal command.
REST API: REST API stands for Representational State Transfer Application Programming Interface. It is a set of rules for building and interacting with web services that use standard HTTP methods to enable communication between client and server.

Footnotes¶

The implementation is actually written in C with many lines of code, applying Lehmer’s algorithm. See also the related GitHub issue.
↩
The interface is served by the code-server app.
↩