Skip to main content

End Criteria

End Criteria are the triggers that determine when a segment of your test sequence has completed successfully. End Criteria can be as simple as waiting for an entity to appear, or as complex as waiting for computer vision to detect a specific object.

Structure of End Criteria​

End Criteria are defined in Bot Segments within the endCriteria field. A segment may contain multiple end criteria, which by default are AND'd together, and therefore must all be true for the segment to pass.

// Segment structure
{
    "name": "Wait for the player to appear",
    "description": "Waits for the player to appear on the screen",
    "endCriteria": [
        {
            "type": ..., // The type of end criteria, as a string
            "transient": ..., // If true, allows future criteria to be evaluated at the same time
            "data": {
                ... // The data for the end criteria, as a JSON object
            }
        }
    ],
    "botAction": {}
}

An end criteria has a type, data, and transient field. The type indicates the type of end criteria to be used, and the data field contains the parameters for that end criteria. If transient is true, this means that this criteria can match at any time during the evaluation of this bot segment and that passing result will persist even if it takes multiple more frames before other criteria in this segment are matched. If transient is false, this means that this criteria and other non-transient criteria must all be true at the same time (any transient criteria must also have matched already).

For example, an end criteria to wait for an entity to appear may look like this:

{
    "name": "Perform an action once the start button appears",
    "description": "Performs an action once the start button appears",
    "endCriteria":[
        {
            "type": "NormalizedPath",
            "transient": false,
            "data": {
                "path": "UI Canvas/Start Button",
                "count": 1,
                "countRule": "GreaterThanEqual"
            }
        }
    ],
    "botAction": {...}
}
info

There are additional fields that can be set on a bot action and many of these inputData fields, such as an apiVersion. You can see the full specification in the source code.

End Criteria Types​

There are many different types of end criteria that can be used in a bot segment, each useful in their own way. The list of supported types are as follows:

  • NormalizedPath: This end criteria waits for a specific game object to exist or not exist in the scene (note: prefer this over PartialNormalizedPath for performance reasons).
  • PartialNormalizedPath: Similar to NormalizedPath, but supports partial matches of the path string (note: this is far less performant than NormalizedPath).
  • UIPixelHash: This end criteria waits and passes if the pixels on the screen change. Currently only supported in games using Coherent GameFace UI.
  • CVText: This end criteria waits for a specific text to appear on the screen using computer vision.
  • CVImage: This end criteria waits for a specific image to appear on the screen using computer vision.
  • CVObjectDetection: This end criteria waits for a specific object to appear on the screen using computer vision.
  • ActionComplete: This end criteria waits for the provided bot action to complete.

There are also two end criteria that allow you to explicitly set And and Or logic between multiple end criteria.

  • And: This end criteria waits for all of the provided end criteria to be true.
  • Or: This end criteria waits for at least one of the provided end criteria to be true.

NormalizedPath​

The NormalizedPath end criteria waits for a specific game object to either exist or not exist in the scene.

Data Format

type is set to NormalizedPath.

// The `endCriteria[].data` format
{
    "path": ..., // The path of the game object to wait for
    "count": ..., // The integer considered by the count rule
    "countRule": ... // The rule to apply to the count, such as "GreaterThanEqual"
}

The countRule can be one of the following:

  • Zero - Indicates that the object should not be present. Ignored the count.
  • NonZero - Indicates that the object should be present. Ignores the count.
  • GreaterThanEqual - Indicates that the object should be present at least the number of times specified by the count.
  • LessThanEqual - Indicates that the object should be present at most the number of times specified by the count.

Example: Waiting for the start button to appear

{
    "name": "Perform an action once the start button appears",
    "description": "Performs an action once the start button appears",
    "endCriteria":[
        {
            "type": "NormalizedPath",
            "transient": false,
            "data": {
                "path": "UI Canvas/Start Button",
                "count": 1,
                "countRule": "GreaterThanEqual"
            }
        }
    ],
    "botAction": {...}
}

PartialNormalizedPath​

The PartialNormalizedPath end criteria waits for a specific game object to either exist or not exist in the scene. It behaves similarly to NormalizedPath, but supports partial matches of the path string by checking if the path string is a substring of the actual path.

Data Format

type is set to PartialNormalizedPath.

// The `endCriteria[].data` format
{
    "path": ..., // The path of the game object to wait for
    "count": ..., // The integer considered by the count rule
    "countRule": ... // The rule to apply to the count, such as "GreaterThanEqual"
}

The countRule can be one of the following:

  • Zero - Indicates that the object should not be present. Ignored the count.
  • NonZero - Indicates that the object should be present. Ignores the count.
  • GreaterThanEqual - Indicates that the object should be present at least the number of times specified by the count.
  • LessThanEqual - Indicates that the object should be present at most the number of times specified by the count.

Example: Waiting for any part of the player model to appear

{
    "name": "Perform an action once the start button appears",
    "description": "Performs an action once the start button appears",
    "endCriteria":[
        {
            "type": "NormalizedPath",
            "transient": false,
            "data": {
                "path": "Player",
                "count": 1,
                "countRule": "GreaterThanEqual"
            }
        }
    ],
    "botAction": {...}
}

UIPixelHash​

The UIPixelHash end criteria waits and passes if the pixels on the screen change. This automatically passes if this is the first frame of the test.

info

This is only supported in games using Coherent GameFace UI.

Data Format

type is set to UIPixelHash.

There is no additional data for this end criteria.

Example: Waiting for the screen to change

{
    "name": "Wait for the screen to change",
    "description": "Waits for the screen to change",
    "endCriteria": [
        {
            "type": "UIPixelHash",
            "transient": false
        }
    ],
    "botAction": {...}
}

CVText​

The CVText end criteria waits for a specific text to appear on the screen using computer vision.

Data Format

type is set to CVText.

  • text - The text to find. Note that this algorithm will treat this as a set of words to find, NOT a sentence. Thus if you have 'Create New Profile' on a single line on one button, but have 'Create' 'New' 'Profile' each on a separate line in different screen area, the algorithm has to choose which one to select. The current method is to select the smallest bounding area containing all required words that is also withinRect (if specified).
  • textMatchingRule - One of Matches or Contains
    • Matches - Each word in the provided text must be found as a whole word in the results. This is a very exact matching rule and can sometimes lead to inconsistent results in frames with poor in game lighting on the text or low text contrast.
    • Contains - Each word in the provided text must be found as a part of a word in the results. This is a looser matching rule and if often used instead of Matches for more stable results. For example "Time" would match to "Timer", "Time", "Times", "Timed", "Time", etc in the frame. This may not seem ideal, but in most game situations, the contrast or text layout isn't always clearly identifiable and extra or incorrect letters may consistently be found for the text you are looking for in a specific frame of the game. When you encounter those situations, you can adjust your game to give better text clarity/contrast to your users and/or you can utilize Contains instead of Matches.
  • textCaseRule - One of Matches or Ignore
    • Matches - (NOT CURRENTLY SUPPORTED - See the Notes section below for details.) The result must match the case of the specified text exactly.
    • Ignore - The specified text is matched without considering capitalization. This option should be used always.
  • withinRect - An optional (can be null/undefined) field to limit the search area to a specific pixel region of the current frame. The SDK will linearly tranform the supplied rect to fit the current resolution using the screenSize as the initial reference resolution.
    • screenSize - The reference resolution in pixels which defines the screen space that rect is defined within.
    • rect - The position (x=0, y=0 is bottom left) and size (width, height) of the rectangle that must contain the supplied text data. The values are defined in pixels.

Notes

The CVText type is still in an experimental phase and may provide inconsistent or unexpected results in many situations.

  • textCaseRule must always be set to Ignore. This method does not currently consider capitalization in its results.
  • Text with low contrast relative to its background may not be detected or may detect incorrect characters
  • Very small or very large text may not be detected or may detect incorrect characters
  • Less common fonts may not be detected or may detect incorrect characters

Example: Waiting for the text "CREATE NEW PROFILE" to appear in a certain region

{
    "name":"CV Text Criteria: Create New Profile Button",
    "endCriteria":[
        {
            "type":"CVText",
            "transient":true,
            "data":{
                "text":"CREATE NEW PROFILE",
                "textMatchingRule":"Contains",
                "textCaseRule":"Ignore"
                "withinRect": {
                    "screenSize":{"x": 1656, "y": 724},
                    "rect":{"x": 900, "y": 110, "width": 350, "height": 50}
                } 
            }
        }
    ],
    "botAction":{}
}

CVImage​

The CVImage end criteria waits for a specific image from a template to appear on the screen using computer vision.

Data Format

type is set to CVImage.

  • imageData - The image data in one of the following formats...
    • The base64 encoded string of the JPG image data - This must be the entire JPG image file, not just the visible bytes.
    • A file:// path to a JPG or PNG image - Be careful when using relative paths as these will be interpreted differently in the Editor vs Runtime builds.
    • A resource:// path to a READABLE Texture2D in one of your project's Resources folders.
    • A resource:// path to a .bytes TextAsset in one of your project's Resources folders that is a JPG or PNG file saved with a .bytes file extension. This can be used if you do not want Unity to import your image as a Texture.
  • withinRect - An optional (can be null/undefined) field to limit the search area to a specific pixel region of the current frame. The SDK will linearly tranform the supplied rect to fit the current resolution using the screenSize as the initial reference resolution.
    • screenSize - The reference resolution in pixels which defines the screen space that rect is defined within.
    • rect - The position (x=0, y=0 is bottom left) and size (width, height) of the rectangle that must contain the supplied image data. The values are defined in pixels.

Notes

The CVImage type is still in an experimental phase and may provide inconsistent or unexpected results in many situations.

  • Multiple matches of the specified image within a frame may provide incorrect or inconsistent result bounds.
  • False positives or incorrect result bounds may occur if pixel regions similar to the specified image exist within a frame.
  • transient should almost always be true for CVImage. CVImage evaluation is a largely asynchronous process that can take multiple frames to complete, thus setting transient=false for the criteria to match within a single frame can lead to situations where the criteria matched on the frame when the request was made, but are no longer matching by the time the result comes back. Setting transient=true allows the needed flexibility for this asynchronous operation to pass more easily, especially when used combination with other endCriteria.

How to get the data for the imageData field as a base64 encoded string

  1. Use a screenshot tool to capture the target area of your game and save this as a JPG file (PNG/BMP/etc are NOT supported at this time).
  2. Run one of the following commands to get the base64 encoded string of the image using the scripts in this directory.
CURRENT_PATH> python encode_jpg_base64.py sample_images/tree.jpg
/9j/4AAQ...<rest of the base64 encoded image data>...VLj3ieS/9k=

or

CURRENT_PATH> ./encode_jpg_base64.sh sample_images/tree.jpg
/9j/4AAQ...<rest of the base64 encoded image data>...VLj3ieS/9k=

Example: Using base64 byte[]

{
    "name":"CV Image Criteria: Menu Change Profile Button",
    "endCriteria":[
        {
            "type":"CVImage",
            "transient":true,
            "data":{
                "imageData":"/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRT/2wBDAQMEBAUEBQkFBQkUDQsNFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBT/wAARCABFAEgDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDE+C37Pc37QzeIZdF+Ki+H73R7iOC60X+wo7p4VaNWSQOZlJVjv5xwVYdq6L/hkWx/6OY0H/wUWf8A8lV85fCn48T/ALM/7U3/AAkrSyDw1e+VZa5bxpv32zxod4X+8jYcEc8MP4iD9OfthfCuHwx4ug8XaNHv8P8AiIfaPOjIMa3DDcwGOgYYcfVsdOPzHLMnq4rM6OAr4lUoVqcZ0n7GjK7UVzwblC7kviWrbW7ud+bZriML7WrT5pcs2mueatq7Ws9uhQ/4ZFsf+jmNB/8ABTZ//JVcV8dP2afF/wALPhe/jjwx8Sbf4g6ZbThL37DpUEQt4uQZdyySBgrbQQORuz0Bx57Xun7LvxIs9H1m/wDBPiJUu/C/iZDayQXJzEsrKV5B4w4Ow+vyelfX5twbjcmwksxoVo4j2VpSpuhRXNBfEk4xTTtdr0seBg+JKmLrKhUlKHNon7Sbs+m77nxJ/wALU8U/9BT/AMl4v/ia+kf2afgP40+PvhHV/FOqeO4/BXh+zlEEN9daVDOlww/1hBLxhVXKjdk5JI7GvO/iJ+yrrvh79pKL4ZaWvmx6rcLNpl02dosnJPmMTz+7VXDepjOOor7I+Ml5pvgPwloHwm8LK1tpWkwxm7KN/rGxlVY9SSSZGz3ZfwylhcLn2JweWZFTpqeIXO5+zhL2dJbyaaau37qTW99jrxGZYrLaFXFYyrO0NEuaS5pdFv8AN+RyC/sZpJ939o7R2+miWp/9uqVv2M1T737R2jr9dEtf/kquZVUs4fQ16p+zL8PV8c+Nm1vUIs6HomJ3dsbHmByiH1AwWPb5cHrz9Pn3h5Q4ey2tmWNzO0aau/8AZ8Pr2S9zduyXmz5TAcaY/McTHD0aTvJ2/iVPv36Hl/xi/Ztn/Z+07R9T1z4r/wBuy6ndG1s9IGgR2zXJ2MzN5gmYqqgZzg8lR/FRXjfxy+PjftLftWRapazySeEtHMun6LExwjRorl58A4zIw3Z67QgPTAK/AsdhcThaeH+vte1nDmaUYxtduytFRV0t3a9/Kx+85HWliKNRptpStq2+i6ts8a+LkIuPHGrIwyCIv/RSV9w/sLfEO1/aI+A+vfA/xNOX8QeH4PP0i8um3sbUufLYc7v3LsEPbY6KPSviL4qf8j7qn/bL/wBFJVL4a/EnVvgr8SdA8daGA9/pM/mNAzFUuIiCskTY7MpI9sg9q/XsVllXMsiw0sLLlr0ownTl2nGKa+T2fSzPzPE1IxzDEQqK8ZSkn6Ns+gtc0W78O6xe6XfwmC8s5mgmjbqrKSCPzFVIVZpBtyCDwRX0r+1DouifEjw14c+MHhC4hvNI1u3jW5MAzhsEK7Y6EEGNgeQyAdeng2j6WZGBK1+wcK5tDifLqWOpx5W9Jx6xmtJRfo/ws+p+Z5pQeWVpUp9Nn3XRn0Z4X/aPt5NJ0671XQft3i6wtJLSDUsLgq23JJ+8N2xCwHXb2zXmt1dTXl3cX95IZrq4kaWSRurMTk1SsLNbWPcRiqOsaoI1Kg19dkHCWTcKSrYnL6PJOrvq31bsrv3Y3bdlZfgfNZnnWOzvko153jDb/N935skka51rUrbTrKNp7u6lWGKNeSzMcAD8TXoH7bPxLt/2aP2e9L+Efh27j/4S3xREx1GaBikkdq2RNKcc/OR5S56qr/3cVu/sw6JpPhnTPEHxa8WXEdloOgQStDLMONyrl5B3OB8ox1ZsDkV+d/xe+K2q/Hj4qa/461jKPqE220tj0trZeIohyei4z6kse9finFeZPiviCOU03fDYRqdTtKr9iHnyL3pebs9j9DyDL1luDeLmvfqaR8o9X8/yG/CCBbfxxpCKMAeZ/wCinoqX4U/8j9pX1l/9FPRX5Tx1/wAjCn/gX/pUj9r4S/3Kf+N/lET4qf8AI+6p/wBsv/RSVyZj875Nu4twABkmus+Kn/I+6p/2y/8ARSV6L+yv8OU8S+MD4k1FMaTobCVd6/JLPglRk/3fvH32+tfpNHHU8tyOjiqivy04WXVvlSSXm3ofA4jDyxWaVaUes5a9ld3fyPoHwT4dufg/+zjp/gy9uHur/UpGuZbZnJS3LuJGVVPQLgDj+Ik1n6Zp6wRgkVqa1qD+JNalvZP9WPkiU9kHT+p/Gs/UL5beMgHFfs/AvD0+H8q9pjP49aTqVOynK2i/wpJfK5+P8UZus2x/Jhv4cFyR80uvzd2Q6pqSwoQDXE6pqRlcgGtPWIdRkszefY7n7H0+0eU3l/8AfWMVyzMWbJr3MbjvbScYPY5MHg/ZJSktz1W4s5/jJ+zZrPgOyvJrTUbM+fHBHKUS4IcyKrAdVY5Ug8ZANfC7Wz2btBIhikjJRkYYKkcEH3r6v8E+KJfCPiK21CPLRj5JUB+8h6j+v1Arhv2qvh7Bo/iSHxbpKodI1s75PKHypORkn6OPm+oavwCphv7BzyrQf8HFt1IvtU+3F+vxL5o/XcPW/tLLozXx0Uotf3fsv5bM81+FP/I/aV9Zf/RT0UfCn/kftK+sv/op6K/M+Ov+RhT/AMC/9KkfpfCX+4z/AMb/ACiWvH+l3WufFC60+yiM13cyQxRRjuxjQD6D3r608O+HrfwL4K0/w1Yk4RN1xL/z0Y8u34n9BiuZ+CfwP1jxp8T9X8QWun/aCDHDZySDCR/ulEkhY9P7oxz96vo6b9mHxjIXcXGlbmOf+Ph/y+5X13DedcP1swwsc5x1KlSwkIS5ZzjFzquKto3tDf8AxWPgeJ6OPwmHrxwVGUquIlJXSb5YX1+ctvQ8WurhbWHANN8DeG5viB4ut9PG5bRT5lzIv8MY6/ieg+tem6n+yb49us+XNpH43T//ABFcR8ZtXuv2Q/g/eiSSM+NNcdrSyktzvCyEH5xkfdjXLcjliB3r9O4p8SsrqYJ4bh7F06+KqNQpxhOMrN/aaTdoxV23tol1PgOGuE61TFqpmVNwpQXNJtWul0Xm3odz/wANpeENY+Ml18EP7NtJPDf9njTI9QQ4U6gpIeA9tu0BARzvBHORjwH4i+DZvA/im706T5oQ2+CT+/GT8p+vr7g18bi3njVbpZ5P7QWT7QLksS/mZ3bs+uec1+hHwtmuP2wPhRp11aPaxeL9KP2W+WVio8wAZJ4JCuMOOvOR2NflOT06Xh9jKdSpUf1SvaNWUnpGr0qtvZT1jLZLRvY/RMdQp5/hKipRtWpNyil1h1j6rdfceM12mjLa+PPBWpeDtUc7ZIy1tJjlCOQR7q2D9M16R/wxT8Qv+euj/wDgW/8A8RU9j+xt8R9PvIrmGbR1kjbcMXb/AJfcr6/iDiLhXOsDLDrNKEaitKEvaw92cdYvf5PybPl8pw+Y5bio1Xh5OD0krPWL3/zXmfEfgXR7rw98VbbTb2MxXVrLNFIp9RG/I9j1B7g0V9K/Gz4Ga34P+IWieJL3Tvszruhu5IzujcGNgjhh3ydvryKK/Cs9zalncsPi6Uk7wSdndKSlK+q6dV5NH7pkWDeBo1KT25rrzTUbf13Pljxl8YviN4b8VahZaL8QvFGi2MTgR2unavcW8SDaDgIjgD8qx/8AhoL4u/8ARWfG/wD4UN5/8door9jwOWYGphKM50INuMfsrsvI/NcZWq/WanvP4n18w/4aC+Lv/RWfG/8A4UN5/wDHazNQ8XeJfHt/DeeKvE2seJrm2UpDLrF9LdNGpOSFMjEgZ7CiivWw+X4OhUVSlRjGXdRSf4I86rVqSg05MfTdN8aeKfAN5cTeFfFOteGJLoL57aPfy2pl2527vLYbsZOM+poorvxFGnXpunVipRfRq6OSjKUJ3i7Gl/w0F8Xf+is+N/8Awobz/wCO0f8ADQXxd/6Kz43/APChvP8A47RRXkf2Tl//AEDw/wDAY/5Hoe2q/wAz+81/CHxm+JPiLxNYWOr/ABF8U6xYyuVktdQ1i5nicbTwUdyD+Iooor814nw1DD4qEaNNRXL0SXVn3WQ1Jyw8uaTev6I//9k=",
                "withinRect": {
                    "screenSize":{"x":1653,"y":714},
                    "rect":{"x":1250,"y":210,"width":250,"height":275}
                } 
            }
        }
    ],
    "botAction":{}
}

Example: Using file:// path (??? represents the path to this folder on your system)

{
    "name":"CV Image Criteria: Menu Change Profile Button",
    "endCriteria":[
        {
            "type":"CVImage",
            "transient":true,
            "data":{
                "imageData":"file://???/sample_images/change_profile_button.jpg",
                "withinRect": {
                    "screenSize":{"x":1653,"y":714},
                    "rect":{"x":1250,"y":210,"width":250,"height":275}
                } 
            }
        }
    ],
    "botAction":{}
}

CVObjectDetection​

The CVObjectDetection end criteria waits for a specific object to appear on the screen using computer vision, based on an image or text query.

Data Format

type is set to CVObjectDetection.

Either imageQuery or textQuery can be specified, you can not use both.

  • imageQuery - The image describing the object to search for in the current frame. The image data must be in one of the following formats...
    • The base64 encoded string of the JPG image data - This must be the entire JPG image file, not just the visible bytes.
    • A file:// path to a JPG or PNG image - Be careful when using relative paths as these will be interpreted differently in the Editor vs Runtime builds.
    • A resource:// path to a READABLE Texture2D in one of your project's Resources folders.
    • A resource:// path to a .bytes TextAsset in one of your project's Resources folders that is a JPG or PNG saved with a .bytes file extension. This can be used if you do not want Unity to import your image as a Texture.
  • textQuery - The string describing the object to search for in the current frame.
  • threshold - An optional (can be null/undefined) field to accept a returned match from the object detection model. Returned matches with a confidence score less than this threshold are ignored. Currently, this is only supported for usage with textQuery.
  • withinRect - An optional (can be null/undefined) field to limit the search area to a specific pixel region of the current frame. The SDK will linearly transform the supplied rect to fit the current resolution using the screenSize as the initial reference resolution.
    • screenSize - The reference resolution in pixels which defines the screen space that rect is defined within.
    • rect - The position (x=0, y=0 is bottom left) and size (width, height) of the rectangle that must contain the supplied image data. The values are defined in pixels.

How to get the data for the imageData field as a base64 encoded string

  1. Find the image you want to use as your query and save it as a JPG file (PNG/BMP/etc are NOT supported at this time).

  2. Use the encode_jpg_base64.py or encode_jpg_base64.sh script in this directory to encode the JPG as a base64 string. The output will be written to STDOUT.

 CURRENT_PATH> python encode_jpg_base64.py sample_images/tree.jpg
/9j/4AAQ...<rest of the base64 encoded image data>...VLj3ieS/9k=

or

 CURRENT_PATH> ./encode_jpg_base64.sh sample_images/tree.jpg
/9j/4AAQ...<rest of the base64 encoded image data>...VLj3ieS/9k=
  1. Create a new bot segment json file and copy the base64 encoded output as the imageQuery of your CVObjectDetection criteria.

Notes The CVObjectDetection type is still in an experimental phase and may provide inconsistent or unexpected results in many situations.

  • Multiple matches of the specified object within a frame will only return one at random.
  • CVObjectDetection via ImageQuery has a very high false positive rate. We are continuing to evaluate and tune this type.
  • CVObjectDetection via ImageQuery selects the most common object in the query image. If the query image contains multiple objects, (such as a cat and a dog) it will only select one--whichever is more prominent.

Example: Image query

{
    "name":"CV Object Detection Criteria: Find a tree using an image query",
    "endCriteria":[
        {
            "type":"CVObjectDetection",
            "description":"Checks for the presence of a tree.",
            "transient":true,
            "data":{
                "imageQuery":"/9j/4AAQ...<rest of the base64 encoded image data>...VLj3ieS/9k=",
                or
                "imageQuery":"file://???/sample_images/tree.jpg",
                or  
                "imageQuery":"resource://sample_images/tree.jpg",
                "withinRect": {
                    "screenSize":{"x":1920,"y":1080},
                    "rect":{"x":1250,"y":210,"width":250,"height":275}
                } 
            }
        }
    ],
    "botAction":{}
}
  • Using file:// path (??? represents the path to this folder on your system)
  • Using resoure:// path (note that the sample_images folder must be under a Resources folder in your project)

Example: Text query

{
    "name":"CV Object Detection Criteria: Find a tree using a text query",
    "endCriteria":[
        {
            "type":"CVObjectDetection",
            "description":"Checks for the presence of a tree.",
            "transient":true,
            "data":{
                "textQuery": "Tree",
                "threshold": 0.4,
            }
        }
    ],
    "botAction":{}
}

ActionComplete​

The ActionComplete end criteria is used to indicate that the segment is completed once the action of this segment has been completed.

Data Format

type is set to ActionComplete.

There is no data field for this end criteria.

Example

{
    "name":"Action Complete Criteria: Action has completed",
    "endCriteria": [
        {
            "type": "ActionComplete",
            "transient": true
        }
    ],
    "botAction":{}
}

And / Or​

The And end criteria waits for all of the provided end criteria to be true. The Or end criteria waits for at least one of the provided end criteria to be true. These can be combined in a nested fashion to create more complex conditions.

Data Format

type is set to And or Or.

// The `endCriteria[].data` format
{
    "criteriaList": [...], // An array of end criteria to AND/OR together
}

Example: Verify that the player is present and that either an enemy or an ally is present

{
    "name": "Verify that the player is present and that either an enemy or an ally is present",
    "endCriteria": [
        { 
            "type": "And", 
            "transient": false,
            "data": { 
                "criteriaList": [
                    { "type": "NormalizedPath", "transient": false, "data": { "path": "Player", "count": 1, "countRule": "GreaterThanEqual" } },
                    { 
                        "type": "Or", 
                        "transient": false,
                        "data": { 
                            "criteriaList": [
                                { "type": "NormalizedPath", "transient": false, "data": { "path": "Enemy", "count": 1, "countRule": "GreaterThanEqual" } },
                                { "type": "NormalizedPath", "transient": false, "data": { "path": "Ally", "count": 1, "countRule": "GreaterThanEqual" } }
                            ] 
                        } 
                    }
                ] 
            } 
        }
    ],
    "botAction":{}
}