Interacting with android application running inside vm

Hey all,

Any suggestions and or tutorials on controlling a smartphone app running in a vm?

My use case is typical shady bot. Drive around pretending to be human clicking on things and liking/upvoting things.

Not really sure where to get started.

One thought was to use selenium to drive a web browser running on the device. I've done this with desktop apps before, and this looks easy enough. There are some work arounds for getting rid of the navigator.webdriver property and spoofing the webgl graphics string is probably also fairly simple.

Another thought would be running the platforms desired smart phone application since it seems like it could be more trusted somehow. This seems a little bit trickier. You could probably use a combination of screen reader to get data back & adb to send input, but I've never done this before.

Anyone have any samples/links to share to help someone find their way?

So you are looking for a selenium equivalent for Android?

There's Appium. It's a lot like Selenium, uses a similar JSON wire protocol like selenium to drive the Android ui.

Ashk881 said:

So you are looking for a selenium equivalent for Android?

There's https://appium.io/. It's a lot like Selenium, uses a similar JSON wire protocol like selenium to drive the Android ui.

Click to expand...

Yes, that seems like it might be a good tool to look at. Thanks for the tip.

I guess the other problem I don't have a good idea how to solve would be how to read what's on the screen and figure out how to interact with programs that aren't mine, such as <social-media-app>.

I feel like you could abuse the accessibility API and use a screen reader somehow, but I'm unsure how to accomplish that or if there are any that you could interact with programmatically.

haywood_jablowme said:

I guess the other problem I don't have a good idea how to solve would be how to read what's on the screen and figure out how to interact with programs that aren't mine, such as <social-media-app>.

I feel like you could abuse the accessibility API and use a screen reader somehow, but I'm unsure how to accomplish that or if there are any that you could interact with programmatically.

Click to expand...

1.You can interact with apps that aren't yours with appium on Android with UiAutomator2/UiAutomator. That's a popular driver that Appium interfaces with for ui automation on Android. All you will need is the name of the `appPackage` , the PATH of the app and an actual Android device with Developer Mode on (not sure how well it works on an Emulator).

2. You don't need to Abuse the Accessibility API to screen-read the contents of some Page. Appium(and UiAutomator)will have access to the HierarchyTree of an AndroidView and you can get the text of an element. All you need is it's XPATH/accessibility Id to find the element.

https://appium.io/docs/en/commands/element/attributes/text/

Having access to the HierarchyTree also means you can trigger events like Clicks, scrolls, swipes etc on element of Views. Just go through the docs for the events you want to interact with.

Here's some:
http://appium.io/docs/en/commands/element/actions/click/
https://appium.io/docs/en/commands/interactions/touch/scroll/