This section introduces the seven types of UIObjects available in Stego.
UIObjects are the basic building blocks used to identify and interact with screen elements during testing.
By dragging a UIObject from the screen into the UIObject Field, you can add and configure it for your scenario.
Detects screen elements like icons and buttons (15+ types) in mobile apps.
Detects numbers and text displayed on the screen.
Finds areas that match a specific image on the screen.
Sets a specific region on the screen.
Selects the entire screen area.
Provides analysis results for secure or protected screens.
Uses OD, OCR, or Crop Image to detect fixed UIObjects, then monitors a region based on their relative position.
OD (Object Detection) #
OD uses buttons, menus, input fields, and other elements recognized by AI in mobile apps as UIObjects.
Inspector #
When selecting an OD UIObject, the Inspector panel displays:
- LABEL: UIObject type
- TEXT: Contained text (if any)
- BOX: Position information on the screen
Attributes #
Click OD in the UIObject Field to configure attributes.
- Label: Defines the UIObject type
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- not used: Ignores the text comparator
- Comparator
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Selector: Defines the order when multiple UIObjects match
- BestMatch selects the most suitable UIObject match. (Default option for Selector.)
- Positive numbers select from the top-left, moving in reading order across the screen (1, 2, 3…).
- Negative numbers select from the bottom-right, starting with the last detected match. (-1, -2, -3…)
OCR (Optical Character Recognition) #
Uses text from UIObjects. Supports English, Korean, numbers, and some special characters.
OCR detects text in word units.
To treat multiple words as one UIObject, drag them together.
Inspector #
Displays OCR-detected UIObject details in the Inspector panel:
- TEXT: Text contained in the UIObject
- BOX: Position of the UIObject on the screen
Attributes #
Click OCR in the UIObject Field to edit attributes.
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- Comparator
- Ignore Line Break
- Applies only with the
=comparator - Treats text as a continuous string, ignoring line breaks when checking for an exact match
- Useful when OCR splits a single sentence into multiple lines
- Applies only with the
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Font Style Sensitivity: Defines whether text matching distinguishes font style
- Selector: Defines the order when multiple UIObjects match
- BestMatch selects the most suitable UIObject match. (Default option for Selector.)
- Positive numbers select from the top-left, moving in reading order across the screen (1, 2, 3…)
- Negative numbers select from the bottom-right, starting with the last detected match. (-1, -2, -3…)
Crop Image #
Use Crop Image to detect a specific image region on the screen.
Drag on the device screen to define the target area.
Excluding unnecessary background improves detection accuracy.
Attributes #
Click Crop Image in the UIObject Field to edit coordinates.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Custom Box #
Defines a specific area on the screen.
Drag on the device screen to set the area.
Use for actions across wide regions such as scrolling or swiping.
Attributes #
Click Custom Box in the UIObject Field to edit coordinates.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Full Screen #
Use the entire screen as a UIObject.
Attributes #
The Full Screen UIObject represents the entire screen area, so its attributes cannot be edited or adjusted.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner
Accessibility #
Use when the screen does not display normally due to security settings, such as a black screen or hidden areas.
Set by selecting a UIObject in the Inspector and dragging it to the field.
Inspector #
Displays information about UIObjects detected through Accessibility.
- IDENTIFIER: Text that distinguishes the UIObject (if provided by the app)
- TYPE: UIObject type
- TEXT: Text contained in the UIObject (if any)
- BOX: UIObject position information on screen
Attributes #
Click Accessibility in the UIObject Field to configure attributes.
- Identifier: Defines the UIObject identifier text (if provided by the app, e.g., ReloadButton)
- Comparator
- =: Matches when the identifier exactly equals the value
- *=: Matches when the Identifier contains the specified value
- ^=: Matches when the Identifier begins with the specified value
- $=: Matches when the Identifier ends with the specified value
- search: Matches when the Identifier satisfies the given regular expression pattern
- not used: Ignores the identifier comparator
- Comparator
- Type: Defines the UIObject type (if provided by the app, e.g., Button)
- Comparator
- =: Matches when the type exactly equals the value
- not used: Ignores the type comparator
- Comparator
- Text: Updates the UIObject text and sets a comparator
- Comparator
- =: Matches when text exactly equals the value
- *=: Matches when text contains the value
- ^=: Matches when text begins with the value
- $=: Matches when text ends with the value
- search: Matches using a regular expression
- not used: Ignores the text comparator
- Comparator
- Text Similarity (Threshold): Applies only with
=comparator- Sets the required similarity for text matching
- Adjust when recognition differs by resolution or environment
- Higher values demand stricter matches (default: 0.8, range: 0–1)
- Case Sensitivity: Defines whether text matching distinguishes case
- Font Style Sensitivity: Defines whether text matching distinguishes font style
- Selector: Defines the order when multiple UIObjects match
- Positive numbers select from the top-left in reading order (1, 2, 3…)
- Negative numbers select from the bottom-right, starting with the last detected match (-1, -2, -3…)
Relative #
Use Relative when you need to define a UIObject based on the position of another reference UIObject.
The relative position follows the reference even when the device resolution or layout changes.
Select a UIObject, then click the Relative button to configure relative positioning.
The reference UIObject appears in yellow, and the Relative area appears in blue.
Note
Relative can be applied to any UIObject type except Full Screen.
Attributes #
Click Relative in the UIObject Field to configure attributes.
Reference UIObject (e.g., OCR): Inherits attributes from the selected reference UIObject type
Relative: Defines the coordinates of the UIObject that exists at a position relative to the reference UIObject.
- Left: X coordinate of the top-left corner
- Top: Y coordinate of the top-left corner
- Right: X coordinate of the bottom-right corner
- Bottom: Y coordinate of the bottom-right corner