This section introduces the seven types of UIObjects available in Stego.
UIObjects are the basic building blocks used to identify and interact with screen elements during testing.
By dragging a UIObject from the screen into the UIObject Field, you can add and configure it for your scenario.
Detects screen elements like icons and buttons (15+ types) in mobile apps.
Detects numbers and text displayed on the screen.
Finds areas that match a specific image on the screen.
Sets a specific region on the screen.
Selects the entire screen area.
Provides analysis results for secure or protected screens.
Uses OD, OCR, or Crop Image to detect fixed UIObjects, then monitors a region based on their relative position.
OD (Object Detection) #
OD uses buttons, menus, input fields, and other elements recognized by AI in mobile apps as UIObjects.
Inspector #
When selecting an OD UIObject, the Inspector panel displays:
- LABEL: UIObject type
- TEXT: Contained text (if any)
- BOX: Position information on the screen
Attributes #
Click OD in the UIObject Field to configure attributes.
- Label: Defines the UIObject type
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- not used: Ignores the text comparator
- Comparator
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Selector: Defines the order when multiple UIObjects match
- Positive numbers select from the top-left, moving in reading order across the screen (1, 2, 3…).
- Negative numbers select from the bottom-right, starting with the last detected match. (-1, -2, -3…)
OCR (Optical Character Recognition) #
Uses text from UIObjects. Supports English, Korean, numbers, and some special characters.
OCR detects text in word units.
To treat multiple words as one UIObject, drag them together.
Inspector #
Displays OCR-detected UIObject details in the Inspector panel:
- TEXT: Text contained in the UIObject
- BOX: Position of the UIObject on the screen
Attributes #
Click OCR in the UIObject Field to edit attributes.
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- Comparator
- Ignore Line Break
- Applies only with the
=comparator - Treats text as a continuous string, ignoring line breaks when checking for an exact match
- Useful when OCR splits a single sentence into multiple lines
- Applies only with the
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Font Style Sensitivity: Defines whether text matching distinguishes font style
- Selector: Defines the order when multiple UIObjects match
- Positive numbers select from the top-left, moving in reading order across the screen (1, 2, 3…)
- Negative numbers select from the bottom-right, starting with the last detected match. (-1, -2, -3…)
Crop Image #
Use Crop Image to detect a specific image region on the screen.
Drag on the device screen to define the target area.
Excluding unnecessary background improves detection accuracy.
Attributes #
Click Crop Image in the UIObject Field to edit coordinates.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Custom Box #
Defines a specific area on the screen.
Drag on the device screen to set the area.
Use for actions across wide regions such as scrolling or swiping.
Attributes #
Click Custom Box in the UIObject Field to edit coordinates.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Full Screen #
Use the entire screen as a UIObject.
Attributes #
Full Screen attributes cannot be changed.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Accessibility #
Use when the screen does not display normally due to security settings, such as a black screen or hidden areas.
Set by selecting a UIObject in the Inspector and dragging it to the field.
Inspector #
Displays information about UIObjects detected through Accessibility.
- IDENTIFIER: Text that distinguishes the UIObject (if provided by the app)
- TYPE: UIObject type
- TEXT: Text contained in the UIObject (if any)
- BOX: UIObject position information on screen
Attributes #
Click Accessibility in the UIObject Field to configure attributes.
- Identifier: Defines the UIObject identifier text (if provided by the app, e.g., ReloadButton)
- Comparator
- =: Matches when the identifier exactly equals the value
- not used: Ignores the identifier comparator
- Comparator
- Type: Defines the UIObject type (if provided by the app, e.g., Button)
- Comparator
- =: Matches when the type exactly equals the value
- not used: Ignores the type comparator
- Comparator
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- not used: Ignores the text comparator
- Comparator
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Font Style Sensitivity: Defines whether text matching distinguishes font style
- Selector: Defines the order when multiple UIObjects match
- Positive numbers select from the top-left in reading order (1, 2, 3…)
- Negative numbers select from the bottom-right, starting with the last detected match (-1, -2, -3…)
Relative #
Use Relative when you need to define a UIObject based on the position of another reference UIObject.
The relative position follows the reference even when the device resolution or layout changes.
Select a UIObject, then click the Relative button to configure relative positioning.
The reference UIObject appears in yellow, and the Relative area appears in blue.
Note: Relative can be applied to any UIObject type except Full Screen.
Attributes #
Click Relative in the UIObject Field to configure attributes.
Reference UIObject (e.g., OCR): Inherits attributes from the selected reference UIObject type
Relative: Defines the coordinates of the UIObject that exists at a position relative to the reference UIObject.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner