Personal computing discussed

Cron and gui applications

Moderators: renee, SecretSquirrel, notfred

Post Reply
 
cheesyking
Minister of Gerbil Affairs
Topic Author
Posts: 2756
Joined: Sun Jan 25, 2004 7:52 am
Location: That London (or so I'm told)
Contact:
Contact cheesyking

Cron and gui applications

Mon Feb 06, 2017 6:49 am

I've got a python script that uses selenium to control a chromium browser window and I'd like to have it run on a schedule on a dedicated VM. The problem I have is that if I use cron to do this I'd have to add something like:
export DISPLAY=:0 && /path/to/command
but this only works if the user is logged in and I'd really like it to run without the user being logged in. Since it's a dedicated VM I suppose I could just run it with a user set to automatically log in but I'd like to at least have looked at doing it some other way.

I've been reading about using xinitrc but this seems a little heavy handed.

Any suggestions for another way of doing this?

(ubuntu 16.04)
Fernando!
Your mother ate my dog!
Top
 
SecretSquirrel
Minister of Gerbil Affairs
Posts: 2726
Joined: Tue Jan 01, 2002 7:00 pm
Location: North DFW suburb...
Contact:
Contact SecretSquirrel

Re: Cron and gui applications

Mon Feb 06, 2017 7:27 am

What part only works if the user is logged in?

One of the environments I'm responsible for at work has a Chromium session that runs in kiosk mode. It runs at system boot as a service and runs as a non-root user. The user isn't logged in and everything works fine. Haven't tried using selenium to interact with it, but interacting with the X display works just fine. You do have to set the DISPLAY environment variable as you note.

--SS
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Cron and gui applications

Mon Feb 06, 2017 7:28 am

What's the application here? Does the window need to be visible or are you just trying to automate some HTTP actions?

If it is the latter (i.e. the window does not need to be visible), you might be better off using a pure CLI solution like wget or cURL instead of automating a browser.
Nostalgia isn't what it used to be.
Top
 
cheesyking
Minister of Gerbil Affairs
Topic Author
Posts: 2756
Joined: Sun Jan 25, 2004 7:52 am
Location: That London (or so I'm told)
Contact:
Contact cheesyking

Re: Cron and gui applications

Mon Feb 06, 2017 8:37 am

SecretSquirrel wrote:
What part only works if the user is logged in?  

Well I tried it from the command line while logged in with ssh. If I have the user logged in to a desktop session then chrome starts and everything works fine. If I don't have the user logged in to the desktop then the python script throws an error (after a long pause) about "chrome failed to start".

just brew it! wrote:
What's the application here? Does the window need to be visible or are you just trying to automate some HTTP actions?

If it is the latter (i.e. the window does not need to be visible), you might be better off using a pure CLI solution like wget or cURL instead of automating a browser.


The window doesn't need to be visible as I'm just trying to automate some http however I spent quite a lot of time looking at pure CLI solutions and the webapp I'm automating seems to be actively hostile towards this. It heavily relies on javascript and everything short of a full browser like I get with selenium seemed completely unworkable. TBH even using selenium was difficult as the names/IDs of many page elements change between each page load.

Just for reference this is downloading call recordings from a voip supplier who doesn't offer automatic ftp transfers or have any kind of API, just a very fiddly website that's often very slow.
Fernando!

Your mother ate my dog!
Top
 
vishalsuvagia
Gerbil In Training
Posts: 8
Joined: Mon Feb 06, 2017 8:18 am

Re: Cron and gui applications

Mon Feb 06, 2017 8:45 am

If you are trying to achieve automation (selemium), which also needs to be run on a regular / periodic basis, It may help with to use build tools like jenkins (jenkins.io) and hudson (hudson-ci.org) .
Using the tools it will not only automatically build the task periodically, you can also view the details of each task, the command being run and the outputs.
This is my TR sign.
Top
 
vishalsuvagia
Gerbil In Training
Posts: 8
Joined: Mon Feb 06, 2017 8:18 am

Re: Cron and gui applications

Mon Feb 06, 2017 9:00 am

cheesyking wrote:
The window doesn't need to be visible as I'm just trying to automate some http however I spent quite a lot of time looking at pure CLI solutions and the webapp I'm automating seems to be actively hostile towards this. It heavily relies on javascript and everything short of a full browser like I get with selenium seemed completely unworkable. TBH even using selenium was difficult as the names/IDs of many page elements change between each page load.

You might not need to login at-all.If you are running selenium tests, you can do the same headlessly ( i.e on a linux without a display ), you will need to set the Display buffer using something like 
Code: Select all
export Xvfb :99,DISPLAY:99 and some additional parameters like screen width and height


You can do a search for linux run browser on headless instance ( as I am a new user, yet unable to post links.)
This is my TR sign.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Cron and gui applications

Mon Feb 06, 2017 9:04 am

Running in a vncserver session is another possibility. A full VM (while it'll get the job done) is overkill.
Nostalgia isn't what it used to be.
Top
 
cheesyking
Minister of Gerbil Affairs
Topic Author
Posts: 2756
Joined: Sun Jan 25, 2004 7:52 am
Location: That London (or so I'm told)
Contact:
Contact cheesyking

Re: Cron and gui applications

Mon Feb 06, 2017 10:46 am

Knew you guys would have some good suggestions. Thanks all I've got it working with xvfb. I'll have to have a look at using something like jenkins in the future and getting vnc working with xvfb so I've got the option of checking what selenium is up to.

xvfb-run --auto-servernum -s "-screen 0 1366x768x24" python /callRecordings/callscraper.py

Spent ages trying to figure out why it wouldn't work with ...1366x768x32... but it turns out that's a known bug.

Thanks again :D
Fernando!

Your mother ate my dog!
Top
 
SecretSquirrel
Minister of Gerbil Affairs
Posts: 2726
Joined: Tue Jan 01, 2002 7:00 pm
Location: North DFW suburb...
Contact:
Contact SecretSquirrel

Re: Cron and gui applications

Mon Feb 06, 2017 1:51 pm

cheesyking wrote:
Knew you guys would have some good suggestions. Thanks all I've got it working with xvfb. I'll have to have a look at using something like jenkins in the future and getting vnc working with xvfb so I've got the option of checking what selenium is up to.

xvfb-run --auto-servernum -s "-screen 0 1366x768x24" python /callRecordings/callscraper.py

Spent ages trying to figure out why it wouldn't work with ...1366x768x32... but it turns out that's a known bug.

Thanks again :D


For a bit of learning.... The reason it wasn't working for you with the normal X server is a permissions thing. The X server started at boot is owned by root. By default, you don't have permission to access the display. Part of the graphical login process grants your account access to the display provided by the system X server. If you did need to see the activity, you could have done the same thing as above with a real X server. The key is that you need to start that X server as the account you want to run the tests. That way you don't have permissions issues.

Glad you got it working.
--SS
Top
 
cheesyking
Minister of Gerbil Affairs
Topic Author
Posts: 2756
Joined: Sun Jan 25, 2004 7:52 am
Location: That London (or so I'm told)
Contact:
Contact cheesyking

Re: Cron and gui applications

Mon Feb 06, 2017 3:07 pm

Thanks again, the graphical side of things in Linux is something I know very little about.
Fernando!

Your mother ate my dog!
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Cron and gui applications

Mon Feb 06, 2017 4:05 pm

cheesyking wrote:
Thanks again, the graphical side of things in Linux is something I know very little about.

The X Window System has an awful lot of legacy baggage; it has been around for more than 3 decades, and was designed while modern GUI concepts were still being developed. Even the basic terminology is confusing at times (e.g. the display being referred to as the "server" and the application being referred to as the "client"). It was designed around the idea that the display was a separate device (an "X terminal") connected over a network to whatever was running the application code; this inevitably results in inefficiencies for the (now more common) use case where the application and display are all running on a single device/system.

The proliferation of DEs and toolkits just adds to the confusion...

There are replacements for X being developed, but it has over 3 decades of inertia behind it at this point, and (even assuming one of the alternatives gets enough traction) de-X-ifying the Linux desktop will likely be a long process.
Nostalgia isn't what it used to be.
Top
 
Redocbew
Minister of Gerbil Affairs
Posts: 2495
Joined: Sat Mar 15, 2014 11:44 am

Re: Cron and gui applications

Mon Feb 06, 2017 4:17 pm

It's a bit self-sustaining now also, because for many people new to the system X is just totally weird, and that just adds to the up-front cost of working on a replacement for it.  I personally never really made it a point of studying too deeply beyond the basics of how X works just because I found many other parts of the system so much more accessible.
Do not meddle in the affairs of archers, for they are subtle and you won't hear them coming.
Top
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: Cron and gui applications

Mon Feb 06, 2017 4:49 pm

Another way to look at it: The X Window System was invented much closer in time to the invention of MS-DOS than to the present day. :wink:
Nostalgia isn't what it used to be.
Top
 
Captain Ned
Global Moderator
Posts: 28704
Joined: Wed Jan 16, 2002 7:00 pm
Location: Vermont, USA

Re: Cron and gui applications

Mon Feb 06, 2017 5:49 pm

just brew it! wrote:
Another way to look at it: The X Window System was invented much closer in time to the invention of MS-DOS than to the present day. :wink:

MS-DOS: August 1981
X: May 1984
What we have today is way too much pluribus and not enough unum.
Top
Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On