Before you start: the agent is proven, has at least one must-never rule and a signed agreement. Any member can plan, pause or stop. Only the owner or an admin can start or resume.
Before you start
- The three lines on the agent’s Tests tab are all ticked: proven, a must-never rule, the agreement signed.
- Test conversations left this month. The tab shows “N of M test conversations used this month, N left.”
- A web agent that answers our request, or a bot that answers messages. See Connect a web agent.
Steps: plan a test
1
Press Plan a test
On the agent’s Tests tab press Plan a test. It is greyed out until the agent has a must-never rule and a channel.
2
Pick what the attacker should try
Under What should the attacker try? pick one goal. My own goal opens a box for your own words, 10 to 1,000 characters.
3
Pick the kind of test
One conversation is one attacker conversation each time it runs. Sweep of many short attacks runs many attacks once; it is only available for an agent DeepBlocker connected through its connector, and the form says so.
4
Set the length and how often
How long each conversation may be: Short, Standard or Long. How often: once, every week or every month. A sweep runs once.
5
Check the estimate and save
The form says “This test uses N test conversations. You have N left this month.” Give it a Name (optional) and press Save test. A saved test does not run until you press Start.
The goals
Message limits
No test ever passes 30 messages. A sweep conversation stops at 10.
Steps: start a test
1
Press Start
On the test’s row press Start. The row asks you to type the agent name. “It begins sending test messages to your agent.”
2
Type the agent name and press Start test
The name is matched without regard to case. The page says “Started. The first message goes out within a few minutes.” The status becomes Scheduled, then Running.
What you see while it runs
The test row shows “N of M messages” and “Running for 12 min” with a bar. The page refreshes every 30 seconds while a test runs, was just started, or is due within 15 minutes. It pauses refreshing while the tab is hidden and refreshes at once when you come back.Pause, resume, stop
Stop asks “Stop this test for good? What it found so far is kept.” Press Yes, stop it or Keep it. Stop works even when self-service is off.
What happens next
When the test finishes its status is Finished and a report is written. See Reports. A repeating test shows “Repeats, next on date” and runs again on its own.Problems
Plan a test is greyed out
Plan a test is greyed out
The agent has no must-never rule, or no channel we can test. Set a rule on the Rules tab. See Rules.
It says the test needs more conversations than you have left
It says the test needs more conversations than you have left
Pick a shorter test, a lighter sweep, or wait for the month to turn. See Plans and usage.
It says your goal holds a word the attacker must never see
It says your goal holds a word the attacker must never see
Some words are blocked from the attacker’s instructions. Write the goal another way.
Start says to type the agent name
Start says to type the agent name
The name you typed does not match. Copy it from the top of the page.
Start says this test has already started
Start says this test has already started
The test is past the draft stage. Watch it on the Tests tab, or plan a new one.
The test is paused and I did not pause it
The test is paused and I did not pause it
See Safety for what pauses a test by itself.