Hello world
Fix a github issue with SWE-agent
In this tutorial, we will fix a github issue with SWE-agent using the command line interface.
- Make sure you have installed SWE-agent and have a language model set up.
- We will be executing code in a Docker sandbox, so make sure you have docker installed (docker troubleshooting). If you cannot run docker, skim this tutorial and see how you can run SWE-agent with cloud-based execution in the command line basics tutorial.
After installing SWE-agent, you have the sweagent command available. Run sweagent --help to see the list of subcommands.
The most important ones are
sweagent run: Run SWE-agent on a single problem statement.sweagent run-batch: Run SWE-agent on a list of problem statements. This is what you would use for benchmarking, or when working with a larger set of historic issues.
In this tutorial, we will focus on the run subcommand.
Let's start with an absolutely trivial example and solve an issue about a simple syntax error (swe-agent/test-repo #1)
sweagent run \
--agent.model.name=claude-3-5-sonnet-20241022 \ # (1)!
--agent.model.per_instance_cost_limit=2.00 \ # (2)!
--env.repo.github_url=https://github.com/SWE-agent/test-repo \
--problem_statement.github_url=https://github.com/SWE-agent/test-repo/issues/1
- We recommend either
Claude 3.5 SonnetorGPT-4ofor this tutorial. - This limits the inference cost per instance to $2. The default is $3.
Annotations
Notice the icon in the right margin in the code snippet? Click on it to display more information about the line.
The example above uses the Claude 3.5 Sonnet model from Anthropic. Alternatively, you can use GPT-4o (from OpenAI).
In order to use it, you need to add your keys to the environment:
export ANTHROPIC_API_KEY=<your key>
export OPENAI_API_KEY=<your key>
alternatively, you can create a .env file in your working directory and put your keys in there like so:
ANTHROPIC_API_KEY=<your key>
OPENAI_API_KEY=<your key>
We should support all models that you can think of. Read more about configuring them here.
Output
INFO This is SWE-agent version 0.7.0 (53c0e1a51aacd6ddfd3eb377a694e150e9b471ff) with SWE-ReX 1.0.1 (deaf1b702fc534a0164eeb4018f7779910a4910c).
🔧 INFO Loading default config from /Users/fuchur/Documents/24/git_sync/SWE-agent/config/default.yaml, because no other config file is specified. Specify --no_config_file to
disable this.
🔧 WARNING demonstration_template is ignored when put_demos_in_history is True
🔧 INFO Setting problem statement based on github issue url
🔧 INFO Loaded environment variables from /Users/fuchur/Documents/24/git_sync/SWE-agent/.env
🔧 WARNING demonstration_template is ignored when put_demos_in_history is True
🏃 INFO Starting environment
🦖 DEBUG Found free port 60661
🦖 INFO Starting container python3.11-0bc626f0-e72a-4faa-9b49-0323d0f1ab87 with image python:3.11 serving on port 60661
🦖 DEBUG Command: "docker run --rm -p 60661:8000 --name python3.11-0bc626f0-e72a-4faa-9b49-0323d0f1ab87 python:3.11 /bin/sh -c 'swerex-remote --auth-token
3f979423-c1fb-40f9-8a44-35256ee60f0a || (python3 -m pip install pipx && python3 -m pipx ensurepath && pipx run swe-rex --auth-token
3f979423-c1fb-40f9-8a44-35256ee60f0a)'"
🦖 INFO Starting runtime at 60661
🦖 INFO Runtime started in 10.21s
🌱 INFO Environment Initialized
🌱 DEBUG Resetting repository SWE-agent__test-repo to commit HEAD
🏃 INFO Running agent
🤠 INFO Trajectory will be saved to
/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-
repo-i1/SWE-agent__test-repo-i1.traj
🦖 DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/registry to /root/tools/registry
🦖 DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmpswoslekq/zipped_transfer.zip
🦖 DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/defaults to /root/tools/defaults
🦖 DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmpf385jdxb/zipped_transfer.zip
🦖 DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/search to /root/tools/search
🦖 DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmppjat9qec/zipped_transfer.zip
🦖 DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/edit_replace to /root/tools/edit_replace
🦖 DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmpiwzg_8ex/zipped_transfer.zip
🦖 DEBUG Uploading file from /Users/fuchur/Documents/24/git_sync/SWE-agent/tools/submit to /root/tools/submit
🦖 DEBUG Created zip file at /var/folders/g3/1y2_hl1j16scls01cxqm76s80000gn/T/tmpdruv408v/zipped_transfer.zip
🧰 INFO Resetting tools
🤠 INFO SYSTEM (main)
SETTING: You are an autonomous programmer, and you're working directly in the command line with a special interface.
The special interface consists of a file editor that shows you 100 lines of a file at a time.
In addition to typical bash commands, you can also use specific commands to help you navigate and edit files.
To call a command, you need to invoke it with a function call/tool call.
Please note that THE EDIT COMMAND REQUIRES PROPER INDENTATION.
For example, if you are looking at this file:
def fct():
print("Hello world")
and you want to edit the file to read:
def fct():
print("Hello")
print("world")
you search string should be `Hello world` and your replace string should be `"Hello"\n print("world")`
(note the extra spaces before the print statement!).
You could also get the same result by search for ` print("Hello world")` and replace with ` print("Hello")\n print("world")`.
RESPONSE FORMAT:
Your shell prompt is formatted as follows:
(Open file: <path>)
(Current directory: <cwd>)
bash-$
First, you should _always_ include a general thought about what you're going to do next.
Then, for every response, you must include exactly _ONE_ tool call/function call.
Remember, you should always include a _SINGLE_ tool call/function call and then wait for a response from the shell before continuing with more discussion and commands.
Everything you include in the DISCUSSION section will be saved for future reference.
If you'd like to issue two commands at once, PLEASE DO NOT DO THAT! Please instead first submit just the first tool call, and then after receiving a response you'll be
able to issue the second .
Note that the environment does NOT support interactive session commands (e.g. python, vim), so please do not invoke them.
🤠 INFO DEMONSTRATION:
/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/demonstrations/replay__marshmallow-code__marshmallow-1867__function_calling_replace__install-1/marshmallow-co
de__marshmallow-1867.traj
🧰 DEBUG Retrieved state from environment: {'open_file': 'n/a', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
We're currently solving the following issue within our repository. Here's the issue text:
ISSUE:
SyntaxError: invalid syntax
I'm running `missing_colon.py` as follows:
```python
division(23, 0)
```
but I get the following error:
```
File "/Users/fuchur/Documents/24/git_sync/swe-agent-test-repo/tests/./missing_colon.py", line 4
def division(a: float, b: float) -> float
^
SyntaxError: invalid syntax
```
INSTRUCTIONS:
Now, you're going to solve this issue on your own. Your terminal session has started and you're in the repository's root directory. You can use any bash commands or the
special interface to help you. Edit all the files you need to and run any checks or tests that you want.
Remember, YOU SHOULD ALWAYS INCLUDE EXACTLY ONE TOOL CALL/FUNCTION CALL PER RESPONSE.
When you're satisfied with all of the changes you've made, you can submit your changes to the code base by simply running the submit command.
Note however that you cannot use any interactive session commands (e.g. python, vim) in this environment, but you can write scripts and run them. E.g. you can write a
python script and then run it with the python command.
NOTE ABOUT THE EDIT COMMAND: Indentation really matters! When editing a file, make sure to insert appropriate indentation before each line!
GENERAL IMPORTANT TIPS:
1. If you run a command and it doesn't work, try running a different command. A command that did not work once will not work the second time unless you modify it!
2. If you open a file and need to get to an area around a specific line that is not in the first 100 lines, say line 583, don't just use the scroll_down command
multiple times. Instead, use the goto 583 command. It's much quicker.
3. If the bug reproduction script requires inputting/reading a specific file, such as buggy-input.png, and you'd like to understand how to input that file, conduct a
search in the existing repo code, to see whether someone else has already done that. Do this by running the command: find_file "buggy-input.png" If that doesn't work,
use the linux 'find' command.
4. Always make sure to look at the currently open file and the current working directory (which appears right after the currently open file). The currently open file
might be in a different directory than the working directory! Note that some commands, such as 'create', open files, so they might change the current open file.
5. When editing files, it is easy to accidentally to write code with incorrect indentation or make other mistakes. Always check the code after you issue an edit to make
sure that it reflects what you wanted to accomplish. If it didn't, issue another command to fix it.
6. When editing files, first explain the code you want to edit and why it is causing the problem. Then explain the edit you want to make and how it fixes the problem.
Explain how the edit does not break existing functionality.
7. Do not try to install any packages with `pip`, `conda`, or any other way. This will usually not work. If the environment is not set up correctly, try to fix the
issue without executing python code or running any tests that require the package installed.
STRATEGY:
1. Always start by trying to replicate the bug that the issues discusses.
If the issue includes code for reproducing the bug, we recommend that you re-implement that in your environment, and run it to make sure you can reproduce the bug.
Then start trying to fix it.
If the bug reproduction script does not print anything when it successfully runs, we recommend adding a print("Script completed successfully, no errors.") command at
the end of the file,
so that you can be sure that the script indeed ran fine all the way through.
2. Locate relevant code using the find and search commands. `open` the file you want to edit.
3. Use the `edit` command to perform edits.
4. When you think you've fixed the bug, re-run the bug reproduction script to make sure that the bug has indeed been fixed.
5. Create additional tests to verify the fix in a style similar to the existing reproduction script. In particular, make sure to test edge cases.
If you find any issues, go back to the file you edited and perform further edits.
(Open file: n/a)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 1 =========================
🤖 DEBUG input_tokens=7,862, output_tokens=36, instance_cost=0.04, cost=0.04
🤖 DEBUG total_tokens_sent=7,862, total_tokens_received=36, total_cost=0.04, total_api_calls=1
🤠 INFO 💭 THOUGHT
Let's start by finding the `missing_colon.py` file that's mentioned in the issue. Based on the error message, it should be in the `tests` directory.
🎬 ACTION
find_file missing_colon.py tests
🧰 DEBUG Retrieved state from environment: {'open_file': 'n/a', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
Found 1 matches for "missing_colon.py" in /SWE-agent__test-repo/tests:
/SWE-agent__test-repo/tests/missing_colon.py
(Open file: n/a)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 2 =========================
🤖 DEBUG input_tokens=7,971, output_tokens=11, instance_cost=0.07, cost=0.04
🤖 DEBUG total_tokens_sent=15,833, total_tokens_received=47, total_cost=0.07, total_api_calls=2
🤠 INFO 💭 THOUGHT
Now let's open the file to see its contents:
🎬 ACTION
open "tests/missing_colon.py"
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/tests/missing_colon.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
[File: tests/missing_colon.py (10 lines total)]
1:#!/usr/bin/env python3
2:
3:
4:def division(a: float, b: float) -> float
5: return a/b
6:
7:
8:if __name__ == "__main__":
9: print(division(123, 15))
10:
(Open file: /SWE-agent__test-repo/tests/missing_colon.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 3 =========================
🤖 DEBUG input_tokens=8,103, output_tokens=48, instance_cost=0.11, cost=0.04
🤖 DEBUG total_tokens_sent=23,936, total_tokens_received=95, total_cost=0.11, total_api_calls=3
🤠 INFO 💭 THOUGHT
I can see the problem. The function definition on line 4 is missing a colon (`:`) after the return type annotation. In Python, all function definitions must end with a
colon. Let's fix this by adding the missing colon.
🎬 ACTION
edit 'def division(a: float, b: float) -> float' 'def division(a: float, b: float) -> float:'
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/tests/missing_colon.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
Text replaced. Please review the changes and make sure they are correct:
1. The edited file is correctly indented
2. The edited file does not contain duplicate lines
3. The edit does not break existing functionality
Edit the file again if necessary.
[File: /SWE-agent__test-repo/tests/missing_colon.py (10 lines total)]
1:#!/usr/bin/env python3
2:
3:
4:def division(a: float, b: float) -> float:
5: return a/b
6:
7:
8:if __name__ == "__main__":
9: print(division(123, 15))
10:
(Open file: /SWE-agent__test-repo/tests/missing_colon.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 4 =========================
🤖 DEBUG input_tokens=8,356, output_tokens=39, instance_cost=0.15, cost=0.04
🤖 DEBUG total_tokens_sent=32,292, total_tokens_received=134, total_cost=0.15, total_api_calls=4
🤠 INFO 💭 THOUGHT
The changes look good. The colon has been added after the return type annotation, and the indentation is correct. Let's test the file by running it with Python to make
sure it works now:
🎬 ACTION
python tests/missing_colon.py
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/tests/missing_colon.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
8.2
(Open file: /SWE-agent__test-repo/tests/missing_colon.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 5 =========================
🤖 DEBUG input_tokens=8,444, output_tokens=37, instance_cost=0.19, cost=0.04
🤖 DEBUG total_tokens_sent=40,736, total_tokens_received=171, total_cost=0.19, total_api_calls=5
🤠 INFO 💭 THOUGHT
The file now runs successfully and produces the expected output (123/15 = 8.2). Let's also test the division with zero case that was mentioned in the issue description:
🎬 ACTION
create test_division.py
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/test_division.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
[File: test_division.py (1 lines total)]
1:
(Open file: /SWE-agent__test-repo/test_division.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 6 =========================
🤖 DEBUG input_tokens=8,535, output_tokens=14, instance_cost=0.23, cost=0.04
🤖 DEBUG total_tokens_sent=49,271, total_tokens_received=185, total_cost=0.23, total_api_calls=6
🤠 INFO 💭 THOUGHT
Let's create a test script that imports and tests the division function:
🎬 ACTION
insert 'from tests.missing_colon import division
try:
result = division(23, 0)
print("This should not be printed - division by zero should raise an error")
except ZeroDivisionError:
print("Test passed: Division by zero correctly raises ZeroDivisionError")
# Test normal case
result = division(10, 2)
print(f"10/2 = {result}")'
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/test_division.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
[File: /SWE-agent__test-repo/test_division.py (12 lines total)]
1:
2:from tests.missing_colon import division
3:
4:try:
5: result = division(23, 0)
6: print("This should not be printed - division by zero should raise an error")
7:except ZeroDivisionError:
8: print("Test passed: Division by zero correctly raises ZeroDivisionError")
9:
10:# Test normal case
11:result = division(10, 2)
12:print(f"10/2 = {result}")
(Open file: /SWE-agent__test-repo/test_division.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 7 =========================
🤖 DEBUG input_tokens=8,754, output_tokens=16, instance_cost=0.27, cost=0.04
🤖 DEBUG total_tokens_sent=58,025, total_tokens_received=201, total_cost=0.27, total_api_calls=7
🤠 INFO 💭 THOUGHT
Let's run the test script to verify both normal division and division by zero:
🎬 ACTION
python test_division.py
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/test_division.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
Test passed: Division by zero correctly raises ZeroDivisionError
10/2 = 5.0
(Open file: /SWE-agent__test-repo/test_division.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 8 =========================
🤖 DEBUG input_tokens=8,731, output_tokens=57, instance_cost=0.31, cost=0.04
🤖 DEBUG total_tokens_sent=66,756, total_tokens_received=258, total_cost=0.31, total_api_calls=8
🤠 INFO 💭 THOUGHT
The tests pass successfully. The function now works as expected:
1. It handles normal division correctly
2. It properly raises a ZeroDivisionError when dividing by zero
3. The syntax error is fixed
Let's clean up our test file since we don't need it anymore:
🎬 ACTION
rm test_division.py
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/test_division.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO 🤖 MODEL INPUT
Your command ran successfully and did not produce any output.
(Open file: /SWE-agent__test-repo/test_division.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO ========================= STEP 9 =========================
🤖 DEBUG input_tokens=8,679, output_tokens=7, instance_cost=0.35, cost=0.04
🤖 DEBUG total_tokens_sent=75,435, total_tokens_received=265, total_cost=0.35, total_api_calls=9
🤠 INFO 💭 THOUGHT
Now we can submit our changes:
🎬 ACTION
submit
🧰 DEBUG Retrieved state from environment: {'open_file': '/SWE-agent__test-repo/test_division.py', 'working_dir': '/SWE-agent__test-repo'}
🤠 INFO Found submission:
diff --git a/tests/missing_colon.py b/tests/missing_colon.py
index 20edef5..5857437 100755
--- a/tests/missing_colon.py
+++ b/tests/missing_colon.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
-def division(a: float, b: float) -> float
+def division(a: float, b: float) -> float:
return a/b
🤠 INFO 🤖 MODEL INPUT
diff --git a/tests/missing_colon.py b/tests/missing_colon.py
index 20edef5..5857437 100755
--- a/tests/missing_colon.py
+++ b/tests/missing_colon.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
-def division(a: float, b: float) -> float
+def division(a: float, b: float) -> float:
return a/b
(Open file: /SWE-agent__test-repo/test_division.py)
(Current directory: /SWE-agent__test-repo)
bash-$
🤠 INFO Trajectory saved to
/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent__test-
repo-i1/SWE-agent__test-repo-i1.traj
╭──────────────────────────── 🎉 Submission successful 🎉 ────────────────────────────╮
│ SWE-agent has produced a patch that it believes will solve the issue you submitted! │
│ Use the code snippet below to inspect or apply it! │
╰─────────────────────────────────────────────────────────────────────────────────────╯
# The patch has been saved to your local filesystem at:
PATCH_FILE_PATH='/Users/fuchur/Documents/24/git_sync/SWE-agent/trajectories/fuchur/default__claude-3-5-sonnet-20241022__t-0.00__p-1.00__c-2.00___SWE-agent__test-repo-i1/SWE-agent
test-repo-i1.patch'
# Inspect it:
cat "${PATCH_FILE_PATH}"
# Apply it to a local repository:
cd <your local repo root>
git apply "${PATCH_FILE_PATH}"
🏃 INFO Done
🌱 INFO Beginning environment shutdown...
🦖 DEBUG Ensuring deployment is stopped because object is deleted
As you can see, the command line options are hierarchical. At the top level, there are three important sections:
problem_statement: What problem are you trying to solve?agent: How do you want to solve the problem? This includes setting up the LM with--agent.model.env: What is the environment in which the problem statement should be solved? This includes setting the repository/folder with the source files with--env.repo, as well as docker images and other dependencies. This will also control where the code is executed (in a local container or in the cloud).
Watching the output, you can notice several stages:
- Setting up the deployment: SWE-agent lets LMs execute actions in sandboxed environments. It can run these environments in docker containers (default), on modal, AWS fargate, or directly on your computer (not recommended). When the deployment starts, you will notice a "starting runtime" message that takes a few seconds. The runtime is what is executing the commands within your deployment.
- Setting up tools: The tools that you specified are copied and installed within the environment.
- System and instance prompts: The initial instructions are shown to the LM.
- Main loop: The LM starts to suggest and execute actions.
- Submission: The LM calls
submitand we extract the patch (i.e., the changes to the source code that solve the problem).
The complete details of the run are saved as a "trajectory" file (more about them here). They can also be turned into new demonstrations together with other log and output files.
Wetted your appetite? Head over to the command line basics tutorial to learn more about the options.