import requests
from PIL import Image
from askui_ml_helper.utils.pta_text import PtaTextInference

pta_text_inference = PtaTextInference("pta-text-v0.1.1.pt")
url = "https://docs.askui.com/assets/images/how_askui_works_architecture-363bc8be35bd228e884c83d15acd19f7.png"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
prompt = 'click on the text "Operating System"'

render_image = pta_text_inference.process_image_and_draw_circle(image, prompt, radius=15)
render_image.show()
>>> Uploaded image with "a red dot", where click operation is predicted 

Get the coordinates

python

import requests
from PIL import Image
from askui_ml_helper.utils.pta_text import PtaTextInference

pta_text_inference = PtaTextInference("pta-text-v0.1.1.pt")
url = "https://docs.askui.com/assets/images/how_askui_works_architecture-363bc8be35bd228e884c83d15acd19f7.png"
image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
prompt = 'click on the text "Operating System"'

coordinates = pta_text_inference.process_image(image, prompt)
coordinates
>>> [0.3981265723705292, 0.13768285512924194]

Contribution

An AskUI's open source initiative. This model is contributed and added to the Hugging Face ecosystem by Murali Manohar @ AskUI.

Citation

TODO

AskUI/pta-text-0.1

Model card for PTA-Text - A Text Only Click Model

Table of Contents

TL;DR

Details for PTA-Text:

Installation

Running the model

Get the annotated image

Get the coordinates

Contribution

Citation

AskUI/
pta-text-0.1