Personal computing discussed

Moderators: renee, farmpuma, just brew it!

 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 4:27 pm

There's been some discussion in the fah forums about clock drift. While the problem isn't solved, it can be circumvented with the suggestions in this thread:
http://foldingforum.org/viewtopic.php?f=44&t=12535

Which lead to this wiki being created:
http://fahwiki.net/index.php/VMware%2C_Linux_Folding_and_Clock_Drift

The workaround involves the use of cron which isn't in the notfreds client, so no help for that at this point.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 4:43 pm

Question: if I just want the VM to fold and not for measuring stuff, do I really need to care about the drift? Just let it quietly chew up the WUs being thrown at the VM is good enough?
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 5:08 pm

Flying Fox wrote:
Question: if I just want the VM to fold and not for measuring stuff, do I really need to care about the drift? Just let it quietly chew up the WUs being thrown at the VM is good enough?

I've seen no effect on folding itself. It just screws up the monitoring programs from what I can tell. I have 15 clients running so having a monitoring program is pretty important to keeping things running.

But I say this knowing that from time to time I would get a crash that says something about the clock being several thousand seconds out of sync. I have not seen one of those since I started this, but they were rare anyway. Generally I would have reset the clock with the date command prior to that.
 
Flying Fox
Gerbil God
Posts: 25690
Joined: Mon May 24, 2004 2:19 am
Contact:

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 6:00 pm

Yeah, it is screwing up measuring the ppd big time I can see. I am seeing it now in FahMon. Another weird thing is that the interval from one % to the next may not be constant. I am getting some % taking 12 minutes and then another take>1hr. :o This is seriously screwing up any measurements.
The Model M is not for the faint of heart. You either like them or hate them.

Gerbils unite! Fold for UnitedGerbilNation, team 2630.
 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 8:20 pm

Flying Fox wrote:
Yeah, it is screwing up measuring the ppd big time I can see. I am seeing it now in FahMon. Another weird thing is that the interval from one % to the next may not be constant. I am getting some % taking 12 minutes and then another take>1hr. :o This is seriously screwing up any measurements.

I moved from fahmon to hfm.net quite a while back, do in large part to fahmon hangs and crashes. I wonder if the clock sync problem is the cause of that. Regardless, both end up with very inaccurate times is the clock is drifting.
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 8:39 pm

I have a script running periodically to do the backups, I could throw in some time sync stuff of some sort in to that.
 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 8:59 pm

notfred wrote:
I have a script running periodically to do the backups, I could throw in some time sync stuff of some sort in to that.

I think that would help a lot of people out.
From what I see it's rarely the hardware clock that drifts, just the system clock, so
hwclock -s
would seem to be a good approach.
I run it every 1 or 2 minutes on machines the loose about 1 minute in 7, but I haven't done any experiments to see what would be optimum.
 
Ragnar Dan
Gerbil Elder
Posts: 5380
Joined: Sun Jan 20, 2002 7:00 pm

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 10:17 pm

Interesting. None of my machines has ever had a huge problem with clock drift, except now and then the VMware-Server-based Ubuntu (6.x or 7.x [kernel 2.6.17-12 I think]) on my main desktop, and then it's usually temporary. A new WU or restarting the client OS has cured it, and it's been rare enough for me that I never cared to get to the bottom of it, especially when my main problem had been process priority for Linux SMP client vs. the GPU client.

Having written that, reading the man page for hwclock certainly goes into the mess that is involved in synchronizing clocks and the consequences. It's good that someone came up with a clever way to make it easy to auto-adjust for reasonable drift values, at least. Having to do it every several minutes would be rather unfortunate. I'd estimate that an adjust every 10 or 15 minutes would be more than enough, even though it might cause problems for monitoring software accuracy.

And that reminds me... I keep forgetting my C2D lost one of its sticks of RAM due to the continuing problem of Crucial Ballistix DDR2 failing on me. I think the stick that failed was actually one of the replacement pair I got from Crucial for an earlier failed pair. The other is now all that's left between that machine and me having to, *gulp*, spend money on it. Except it's elderly and I don't wish to spend on it any more. I should probably sell it (the C2D 6400, the Gigabyte 965P mobo it's running on, and the RAM, after getting another free replacement from Crucial for the next owner to concern himself with), get rid of another low-performance SMP folder (X2-3800), and replace them with an 8 core machine ready to try -bigadv WU's. But the -passkey requirement irks me, and I'm going to have to do power calculations which will be difficult to find accurate figures for in order to compute them. At least I'll get an estimate of whether it's worth the effort and what the electrical costs or savings would be (because I might retire other power-hungry producers too, depending on output).

I'm also reminded now would probably be a good time to start up my months-old "new" Ubuntu 9.x install and make sure it's got all its updates. Eventually it will get my current SMP client, if I quit reading about performance and other problems for a long enough time.

Sorry for the wordiness, but I'm a bit more relaxed than usual thanks to consuming a bit more alcohol than usual for a weeknight. :wink:
 
JPinTO
Gerbil Team Leader
Posts: 240
Joined: Sat Jun 30, 2007 7:02 am
Location: Toronto, Ontario

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 10:32 pm

If you install Vmware tools onto Linux, there is a synchronize time with host option.

- JP
 
Kougar
Minister of Gerbil Affairs
Posts: 2306
Joined: Tue Dec 02, 2008 2:12 am
Location: Texas

Re: VMWare - Linux Folding Client & Clock Drift

Thu Dec 24, 2009 10:48 pm

This may be a bit silly to ask, but I presume clock sync issues weren't from HPET being incorrectly set to 32bit when the host OS (and/or guest OS) were 64bit?

HPET has been supported since Vista / Windows 7 but few people mention it needs to be configured to "64bit" mode when running 64bit OS's, be it Linux or Windows. I do know HPET is supposed to be much more precise than older clock timers, but still I've (rarely) seen timing issues arise on multicore AMD platforms when HPET was set incorrectly to 32bit.... typically almost always with CPU overclocks.

JPinTO wrote:
If you install Vmware tools onto Linux, there is a synchronize time with host option.

- JP


Interesting, I'll have to see if VirtualBox offers the same with their equivalent tool set.
 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

Re: VMWare - Linux Folding Client & Clock Drift

Fri Dec 25, 2009 12:04 am

Ragnar Dan wrote:
But the -passkey requirement irks me, and I'm going to have to do power calculations which will be difficult to find accurate figures for in order to compute them. At least I'll get an estimate of whether it's worth the effort and what the electrical costs or savings would be (because I might retire other power-hungry producers too, depending on output).

Sorry for the wordiness, but I'm a bit more relaxed than usual thanks to consuming a bit more alcohol than usual for a weeknight. :wink:


It sounds like we'll all be stuck with the passkey eventually. From what I've been reading at the fah forum the bigadv client is just the first to start rewarding people for early return of work units, and the passkey will be required for that. 7im posted a message today that we should start now and make sure we can make it work.

tis the sesson for the toddy....cheers :lol:
 
weedacres
Gerbil
Topic Author
Posts: 18
Joined: Thu Nov 12, 2009 8:37 pm

Re: VMWare - Linux Folding Client & Clock Drift

Fri Dec 25, 2009 12:09 am

JPinTO wrote:
If you install Vmware tools onto Linux, there is a synchronize time with host option.

- JP

VMtools won't install on notfreds nor the bigadv slackware client. I think you need a full blown linux distribution for it to install.
When I was running Ubuntu 8.04 under VMware I had vmtools installed and it didn't seem to help with the clock drift.
 
notfred
Maximum Gerbil
Posts: 4610
Joined: Tue Aug 10, 2004 10:10 am
Location: Ottawa, Canada

Re: VMWare - Linux Folding Client & Clock Drift

Fri Dec 25, 2009 3:59 pm

You are always going to have horrible clock hacks with virtualisation due to the way clock cycles get allocated to the guest OS. My folding stuff normally runs NTP but it gets in to a big mess when virtualised so I detect various VMs and disable NTP if found. The VM is meant to do time sync to the guest but it apparently is one of those areas that is soft in many VMs.
 
JPinTO
Gerbil Team Leader
Posts: 240
Joined: Sat Jun 30, 2007 7:02 am
Location: Toronto, Ontario

Re: VMWare - Linux Folding Client & Clock Drift

Fri Dec 25, 2009 8:32 pm

I've got a half dozen Ubuntu's under VMWare, and I haven't had any trouble having them keep time with VMWare tools installed and the "synchronize time with host" option enabled. Oddly, it is disabled by default.

- JP
 
just brew it!
Administrator
Posts: 54500
Joined: Tue Aug 20, 2002 10:51 pm
Location: Somewhere, having a beer

Re: VMWare - Linux Folding Client & Clock Drift

Fri Dec 25, 2009 10:47 pm

The following script may be helpful. It aggressively (every 10 seconds) attempts to resynchronize the clock. You need to have the ntpdate tool installed, and you need to have a local NTP server set up (on your LAN), since it is rather rude to hit a public NTP server every 10 seconds.
#!/bin/bash
# timesync - Aggressive clock synchronizer for VirtualBox VMs
# Must be run as root!
# Specify the name or IP address of the NTP server to sync with on the command line.
# If you specify an option of -b, the script will run in the background instead of in the
# foreground.

checkinterval=10           # check interval (secs)
slewthresh=25000           # threshold before we slew (usecs)
stepthresh=250000          # threshold before we step (usecs)
slewdelay=300              # min time between successive slew requests (secs)
logfile=/var/log/timesync  # log file for background operation

# If -b specified, go to background
if [[ $1 == '-b' ]]; then
    shift
    $0 $* > $logfile 2>&1 &
    exit
fi

server=$1
if [[ "$server" == "" ]]; then
    echo No NTP server specified! >&2
    exit 1
fi

echo "`date`: timesync started (server=$server)"

lastslew=0
while true; do
    offset=$((`ntpdate -q $server | head -n1 | sed -e 's/^.*offset //g;s/,.*$//g;s/-//g;s/\.//g;s/^0*//g'`))
    if [[ $offset -ge $slewthresh ]]; then
        if [[ $offset -ge $stepthresh ]]; then
            ntpdate -b $server
        else
            now=`date +%s`
            elapsed=$(($now-$lastslew))
            if [[ $elapsed -ge $slewdelay ]]; then
                ntpdate $server
                lastslew=$now
            fi
        fi
    fi
    sleep $checkinterval
done

I've also found that futzing with the clocksource= option on the kernel boot line can sometimes help. Displaying the contents of the file /sys/devices/system/clocksource/clocksource0/available_clocksource will tell you which clock sources are available; try each one in turn to find the one which gives you the least clock drift.
Nostalgia isn't what it used to be.

Who is online

Users browsing this forum: No registered users and 1 guest
GZIP: On