Re: [MSNoise] MSNoise Digest, Vol 7, Issue 14
by Wardhani, Prima Wira Kusuma
Sorry being late in reply,,,
I don't know that you are reply my email, I found it in
https://www.mail-archive.com/msnoise@mailman-as.oma.be/msg00158.html
after search for error "S07plot_dtt.py", it didn't receive in my email
in any folder. I think you didn't reply my email,, so i tried to solve
it myself,, :(
1. How do you calculate/get the RAM/CPU values ?
Ans : I used trial and error by running the script, and wait for
its to finish.
2. What OS do you work on ?
Ans : I work both in Linux (CentOS) and Windows 8.1. I tried to run
the script in different OS.
3. Which Python version/architecture do you use ? (use bugreport.py -a
and send it to the list) - Are you sure your machine is a quadcore ?
or could it be a dual core,capable of hyperthreading (2x2) ?
Ans : I use Anaconda with python 2.7, numpy 1.8.1, obspy 0.9.2,
matplotlib 1.3.1, as you can see in the bugreport.py result. I'm sure
my computer is Quadcore i7 3630QM, it hypertreading to 4x2 = 8, and I
already tested in another parallel program option with my GTX 600 GPU
series.
************* Computer Report *************
----------------+SYSTEM+-------------------
Windows
Wardhani
8
6.2.9200
AMD64
Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
----------------+PYTHON+-------------------
Python: 2.7.6 |Anaconda 2.0.0 (64-bit)| (default, Nov 11 2013,
10:49:15) [MSC v.1500 64 bit (AMD64)]
---------------+MODULES+-------------------
Required:
[X] numpy: 1.8.1
[X] scipy: 0.14.0
[X] pandas: 0.13.1
[X] matplotlib: 1.3.1
[X] statsmodels: 0.5.0
[X] sqlalchemy: 0.9.4
[X] traitsui: 4.4.0
[X] traits: 4.4.0
[X] enable: 4.3.0
[X] scikits.samplerate: present (no version)
[X] obspy: 0.9.2
[X] sphinx: 1.2.2
[X] jinja2: 2.7.2
Backends: (at least one is required)
[X] wx: 3.0.0.0
[X] PyQt4: present (no version)
[X] PySide: 1.2.1
Not required, just checking:
[X] setuptools: 3.6
[X] reportlab: $Id$
[X] configobj: 5.0.5
[X] pkg_resources: present (no version)
[X] paramiko: 1.14.0
[X] ctypes: 1.1.0
[X] pyparsing: 2.0.1
[X] distutils: 2.7.6
[X] IPython: 2.1.0
[X] vtk: present (no version)
4. Are you using a MySQL server ? On your local machine or on a remote one ?
Ans : I use local domain 127.0.0.1, MySQL server using WAMP or XAMPP
(I tried both of this) in windows 8.1, and use MariaDB + httpd in
CentOS.
Thank you for your time
Regards,,
Dear Prima,
First, (the easy answer), the fact that the GPU stays at 0% is
perfectly normal,
as MSNoise doesn't use GPU calculation (or Cuda, or anything related to
GPUs).
The increase between the -t 1 and -t 2 seems logical in terms of timing. But
from 2 to 4, you don't seem to gain any timing increase.
So:
- How do you calculate/get the RAM/CPU values ?
- What OS do you work on ?
- Which Python version/architecture do you use ? (use bugreport.py -a and
send it to the list) - Are you sure your machine is a quadcore ? or could
it be a dual core,capable of hyperthreading (2x2) ?
- Are you using a MySQL server ? On your local machine or on a remote one ?
Scanning the archive actually doesn't consume much power, but requires quite
a large number of readings, so your disk performance could be an issue too.
I haven't yet played a lot with Numba, it's a thing to check in the future.
If you succeed in doing that, PR (pull requests) are welcome ! This said,
there are few chances the first steps (scan + jobs) can be accelerated
dramatically, as the first mainly relies on Obspy and the second on the
power of your database machine.
Best,
Thomas
Le 17/05/2014 13:25, Wardhani, Prima Wira Kusuma a écrit :
Dear Thomas,
Dear All,
I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise
using CPU-GPU-RAM computing.
I processed 49 seismic station, with 2 months data available. For scanning
the archive *s01scan_archive.py*, using --init option, and thread option :
--init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours
--init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
--init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours
the next threads option RAM comsume stuck in 1.6GB, no increase processing
using multicore CPU.
After finish I tried create new jobs using *s02new_jobs.py* for all
station, and it
takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the
process only consume 1.6GB RAM, 17% CPU, no more than this value.
How to optimalize the Processing using RAM-CPU-and GPU? Because it takes
along time to process all my Data,, at least it can be optimize using
RAM-CPU it will be better. My Computer using 16GB RAM and 4Core(a)3.2GHz, but
it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason,
it failed,.
Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize
MSNoise code?
Thank you,
WARDHANI, Prima Wira Kusuma
Master of Earthquake Engineering and Engineering Seismology
Universite Joseph Fourier - ISTerre Room 015prima.dhan...(a)gmail.com
+62-81548815815
+33-663964113
がんばります！
_______________________________________________
MSNoise mailing
listMSNoise@mailman-as.oma.behttp://mailman-as.oma.be/mailman/listinfo/msnoise
