17 May
2014
17 May
'14
1:25 p.m.
Dear Thomas,
Dear All,
I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise
using CPU-GPU-RAM computing.
I processed 49 seismic station, with 2 months data available. For scanning
the archive *s01scan_archive.py*, using --init option, and thread option :
--init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours
--init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
--init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
the next threads option RAM comsume stuck in 1.6GB, no increase processing
using multicore CPU.
After finish I tried create new jobs using *s02new_jobs.py* for all
station, and it
takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the
process only consume 1.6GB RAM, 17% CPU, no more than this value.
How to optimalize the Processing using RAM-CPU-and GPU? Because it takes
along time to process all my Data,, at least it can be optimize using
RAM-CPU it will be better. My Computer using 16GB RAM and 4Core(a)3.2GHz, but
it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason,
it failed,.
Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize
MSNoise code?
Thank you,
WARDHANI, Prima Wira Kusuma
Master of Earthquake Engineering and Engineering Seismology
Universite Joseph Fourier - ISTerre Room 015
prima.dhani06(a)gmail.com
+62-81548815815
+33-663964113
がんばります！
19 May
19 May
10:36 a.m.
Dear Prima,
First, (the easy answer), the fact that the GPU stays at 0% is perfectly
normal, as MSNoise doesn't use GPU calculation (or Cuda, or anything
related to GPUs).
The increase between the -t 1 and -t 2 seems logical in terms of timing.
But from 2 to 4, you don't seem to gain any timing increase.
So:
- How do you calculate/get the RAM/CPU values ?
- What OS do you work on ?
- Which Python version/architecture do you use ? (use bugreport.py -a
and send it to the list)
- Are you sure your machine is a quadcore ? or could it be a dual core,
capable of hyperthreading (2x2) ?
- Are you using a MySQL server ? On your local machine or on a remote one ?
Scanning the archive actually doesn't consume much power, but requires
quite a large number of readings, so your disk performance could be an
issue too. I haven't yet played a lot with Numba, it's a thing to check
in the future. If you succeed in doing that, PR (pull requests) are
welcome ! This said, there are few chances the first steps (scan + jobs)
can be accelerated dramatically, as the first mainly relies on Obspy and
the second on the power of your database machine.
Best,
Thomas
Le 17/05/2014 13:25, Wardhani, Prima Wira Kusuma a écrit :
Dear Thomas,
Dear All,
I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise
using CPU-GPU-RAM computing.
I processed 49 seismic station, with 2 months data available. For scanning
the archive *s01scan_archive.py*, using --init option, and thread option :
--init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours
--init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
--init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
the next threads option RAM comsume stuck in 1.6GB, no increase processing
using multicore CPU.
After finish I tried create new jobs using *s02new_jobs.py* for all
station, and it
takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the
process only consume 1.6GB RAM, 17% CPU, no more than this value.
How to optimalize the Processing using RAM-CPU-and GPU? Because it takes
along time to process all my Data,, at least it can be optimize using
RAM-CPU it will be better. My Computer using 16GB RAM and 4Core(a)3.2GHz, but
it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason,
it failed,.
Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize
MSNoise code?
Thank you,
WARDHANI, Prima Wira Kusuma
Master of Earthquake Engineering and Engineering Seismology
Universite Joseph Fourier - ISTerre Room 015
prima.dhani06(a)gmail.com
+62-81548815815
+33-663964113
がんばります！
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
4002
days inactive
4004
days old
1 comments
2 participants
participants (2)
-
Thomas Lecocq -
Wardhani, Prima Wira Kusuma