MSNoise

msnoise@lists.observatory.be

349 discussions

Some little things and one question about data from more than one network.

by Kasper van Wijk

Dear Tom et al., I am working with a student to replicate the install of MSNOISE for the Auckland Volcanic Field. Below are a few things we have ran into: 1. http://msnoise.org/doc/workflow.html#module-MSNoise.s001configurator misses the explicit call to start the configurator: python s001configurator.py (this is obvious to anyone who has used MSNOISE before, but stumped the aforementioned new student) 2. On CENTOS linux, I have an issue with sqlalchemy. The "fix" is as ugly as it is mysterious: http://stackoverflow.com/questions/14600530/cant-import-from-sqlalchemy-ext… 3. We have a font problem (which I don't know how to fix), but starting the configurator, we get the attached warning. Just a warning, though. 4. Is it correct that sqlite can only work on one thread? [devora@localhost MSNoise-master]$ python s01scan_archive.py --init -t 2 2014-08-22 10:14:13,724 [INFO] *** Starting: Scan Archive *** 2014-08-22 10:14:13,727 [INFO] Initializing (should be run only once) 2014-08-22 10:14:13,727 [INFO] You can not work on 2 threads because SQLite only supports 1 connection at a time 2014-08-22 10:14:13,727 [INFO] Will work on 1 threads 5. And this is where are currently: we imported a station into our NZ network with a different network name. This resulted in errors when pairing the AK station with an NZ station. What should we do? After getting an error, we tried renaming the AK data to NZ to be consistent, but we are unable to cleanly perge the AK info in the database. Alternatively, can we set up two networks in the configurator? Cheers, kasper

10 years, 7 months

MSNoise releases - Development

by Thomas Lecocq

Dear MSNoise users, You might have noticed I am working on the codes, and updating the "branches" of the GitHub repository. I just wanted to remind you all that, except if you're really reckless, you should only use "Releases" versions of the code. The releases are available here : https://github.com/ROBelgium/MSNoise/releases to be sure, always download the "Lastest" : https://github.com/ROBelgium/MSNoise/releases/latest The branches (tests, msnoise-admin, etc) and the 'master' branch are often updated with code that is not yet fully tested or complete, and so could not work. I have started writing unit tests for the code, so, in a near future, one could just look at the "Build Status" of TravisCI (CI stands for Continuous Integration) to check if all is OK to test the "master" instead of "Releases" ! Best regards from Brussels Thomas ps: as always, if you use MSNoise, please let me/us know! Certainly if you had/have problems ! If you think something would be useful for others, submit ideas/issues/suggestions to the https://github.com/ROBelgium/MSNoise/issues ! -- Dr. Thomas Lecocq Geologist Seismology - Gravimetry Royal Observatory of Belgium * * * * * * * * * * --------- http://www.seismology.be http://twitter.com/#!/Seismologie_be

10 years, 7 months

(no subject)

by Aurélien Mordret

Greetings Thomas, greetings dear MSNoise users, First of all, I would like to thank the authors for their invaluable contribution: MSNoise is a really nice piece of work!! Congrats! Now let me start with my questions: I'm computing correlations from LHZ components (1 Hz sampling frequency), I'm asking for 1Hz output correlations so I put the re-sampling parameter at 1 Hz. I correlate chunks of data 14400 s long (4 hours). 1) I obtain this annoying warning in the whiten.py function (I show just one example, but it happens at every lines involving the parameter 'high', lines 86,100,102) ./MSNoise-1.2.4/whiten.py:102: DeprecationWarning: using a non-integer number instead of an integer will result in an error in the future FFTRawSign[high:-high] *= 0 I understand that is because 'high' is an indice and should be an integer, but I don't really understand why it is NOT an integer in my case because it seems to me that this problem has been assessed in the s03compute_cc.py function(line 380-382). Is it because, line 398-399, in the function whiten, Nfft should be replaced by int(Nfft)? Beside the bunch of messages on the screen, it does not affect the correlation computation. 2) Some times to times, when I'm computing the correlations, the function exits with an error when trying to gather daily files which have not the same type (I don't know what that means...) Traceback (most recent call last): File "s03compute_cc.py", line 214, in <module> stream.merge(fill_value=0) File "/usr/local/lib/python2.7/dist-packages/obspy/core/stream.py", line 1715, in merge self._mergeChecks() File "/usr/local/lib/python2.7/dist-packages/obspy/core/stream.py", line 1660, in _mergeChecks raise Exception(msg) Exception: Can't merge traces with same ids but differing data types! I checked the file where it seems there is a problem, but it looks OK. My solution so far is just to jump to the next job by re-running the correlation function but it's annoying to have to check regularly if the run wasn't aborted. I know it's more an obspy problem than a MSNoise one, but, does anyone faced this problem before? Thanks, Aurelien -- ========================== Aurelien Mordret Postdoctoral fellow Department of Earth, Atmospheric and Planetary Sciences Massachusetts Institute of Technology 54-526 Cambridge, MA 02139-4307 e-mail: mordret(a)mit.edu web: http://web.mit.edu/mordret/www

10 years, 7 months

problem running s001configurator.py

by Virginie Dhour

Hello, I'm trying to install MSNoise but when I run s001configurator.py, I got the following error: virginie@PC-Virginie:~/MSNoise-1.2.4$ python s001configurator.py Cannot mix incompatible Qt library (version 0x40802) with this library (version 0x40805) Abandon I found on internet that I might have two versions of Qt library (I use debian). Unfortunatly, I don't manage to sort it out. Did it happen to one of you? If yes, what have you done to fix it? Thanks in advance. Virginie

10 years, 10 months

dtt_plot issue (NameError: name 'left' is not defined)

by Dimitris Giannopoulos

Dear all, I have a problem when I am trying to make the dtt plot. I have the following error: _File "s07plot_dtt.py", line 112, in <module>_ _ plt.xlim(left, right)_ _NameError: name 'left' is not defined_ I have seen in a previous message in the mailing list something about that. Xiao, if I remember well, had mentioned that if mov_stack is not set up 1, then left and right are not defined. Does anybody know what I have to do about that? Thank you very much in advance, Dimitris >From prima.dhani06(a)gmail.com Wed Jun 4 18:16:16 2014 Return-Path: <prima.dhani06(a)gmail.com> X-Original-To: msnoise(a)mailman-as.oma.be Delivered-To: msnoise(a)mailman-as.oma.be Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com [209.85.217.177]) by mailman-as.oma.be (Postfix) with ESMTP id A3BD1233B9 for <msnoise(a)mailman-as.oma.be>; Wed, 4 Jun 2014 18:16:16 +0000 (UTC) Received: by mail-lb0-f177.google.com with SMTP id s7so4472673lbd.22 for <msnoise(a)mailman-as.oma.be>; Wed, 04 Jun 2014 11:15:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=4gazFLRQ9oK6lyO+IA5heOiO7J3DCFjKd5gFhMS0pyM=; b=RqWc/v+RJhec6JbBrDAhf0KEGbWC1BfzWoFwjyVOs/y04ExyQRo3aITlrbqAHcSNUB LnKY0Vqyn1rG4NdGWfUK72Atw2+W1zpOrFfgUOYQNMLz5w+3TcH3hKIk7rwTUEyuyHc1 bQTH5TyEb71NfRZHQwHZpBRX99P9Vs9wcU88d3A1PbbhppAIYDAgduKPpatL70b1xi6w a91bDFnCDK8wqTbofLpOFsRunlXs7baFOrDbB/8hm+atWKYmmY/vpCeuakubrXYAcqVc 8mRgD/PiBLL1/z+/ElAvwCtt0E3AV1ioIqHp/iFraN2hjPYM1W1gp47EcZiEcrpWdLms E9EQ== MIME-Version: 1.0 X-Received: by 10.112.85.202 with SMTP id j10mr3668501lbz.95.1401905754784; Wed, 04 Jun 2014 11:15:54 -0700 (PDT) Received: by 10.152.106.3 with HTTP; Wed, 4 Jun 2014 11:15:54 -0700 (PDT) Date: Wed, 4 Jun 2014 20:15:54 +0200 Message-ID: <CALmBX8NbQDGr1_DRPJqzdQ32CH9QjthopE6nJd4Jp3z3Tqa25A(a)mail.gmail.com> From: "Wardhani, Prima Wira Kusuma" <prima.dhani06(a)gmail.com> To: msnoise(a)mailman-as.oma.be Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.12 Subject: [MSNoise] dtt_plot issue + PLOT=TRUE issue in compute_dtt X-BeenThere: msnoise(a)mailman-as.oma.be X-Mailman-Version: 2.1.12 Precedence: list Reply-To: Python Package for Monitoring Seismic Velocity Changes using Ambient Seismic Noise <msnoise(a)mailman-as.oma.be> List-Id: Python Package for Monitoring Seismic Velocity Changes using Ambient Seismic Noise <msnoise.mailman-as.oma.be> List-Unsubscribe: <http://mailman-as.oma.be/mailman/options/msnoise>, <mailto:msnoise-request@mailman-as.oma.be?subject=unsubscribe> List-Archive: <http://mailman-as.oma.be/pipermail/msnoise/> List-Post: <mailto:msnoise@mailman-as.oma.be> List-Help: <mailto:msnoise-request@mailman-as.oma.be?subject=help> List-Subscribe: <http://mailman-as.oma.be/mailman/listinfo/msnoise>, <mailto:msnoise-request@mailman-as.oma.be?subject=subscribe> X-List-Received-Date: Wed, 04 Jun 2014 18:16:16 -0000 Dear Thomas, Dear All, I have a problem when I am trying to make the dtt plot in "s07plot_dtt", same as Dimitris, : Traceback (most recent call last): File "s07plot_dtt.py", line 112, in <module> plt.xlim(left, right) NameError: name 'left' is not defined in first ZZ correlation, then I have : Traceback (most recent call last): File "07.plot_dtt.py", line 148, in <module> plt.savefig('dtt_allmovstacksNEW_%s.png' % dttname, dpi=300) NameError: global name 'dttname' is not defined in RR correlation... This two result have diferrent problems, but I used same parameter. All of this data already stacked by 1,3,7, then MWCS, and DTT. All folder and all data present correspending with the configuration start and end ref. And i tried to plot delay, error, phase coherrence, data selection, delay time variation using "PLOT = TRUE" option in "s06compute_dtt" but the process only got 1 day, then the process got an error, unfinished job for DTT. but if i used "PLOT = FALSE", all DTT job is done. How to solve this problems? Thank you for your time, Regards WARDHANI, Prima Wira Kusuma Universite Joseph Fourier prima.dhani06(a)gmail.com $B$,$s$P$j$^$9!*(B

10 years, 10 months

Re: [MSNoise] MSNoise Digest, Vol 7, Issue 14

by Wardhani, Prima Wira Kusuma

Sorry being late in reply,,, I don't know that you are reply my email, I found it in https://www.mail-archive.com/msnoise@mailman-as.oma.be/msg00158.html after search for error "S07plot_dtt.py", it didn't receive in my email in any folder. I think you didn't reply my email,, so i tried to solve it myself,, :( 1. How do you calculate/get the RAM/CPU values ? Ans : I used trial and error by running the script, and wait for its to finish. 2. What OS do you work on ? Ans : I work both in Linux (CentOS) and Windows 8.1. I tried to run the script in different OS. 3. Which Python version/architecture do you use ? (use bugreport.py -a and send it to the list) - Are you sure your machine is a quadcore ? or could it be a dual core,capable of hyperthreading (2x2) ? Ans : I use Anaconda with python 2.7, numpy 1.8.1, obspy 0.9.2, matplotlib 1.3.1, as you can see in the bugreport.py result. I'm sure my computer is Quadcore i7 3630QM, it hypertreading to 4x2 = 8, and I already tested in another parallel program option with my GTX 600 GPU series. ************* Computer Report ************* ----------------+SYSTEM+------------------- Windows Wardhani 8 6.2.9200 AMD64 Intel64 Family 6 Model 58 Stepping 9, GenuineIntel ----------------+PYTHON+------------------- Python: 2.7.6 |Anaconda 2.0.0 (64-bit)| (default, Nov 11 2013, 10:49:15) [MSC v.1500 64 bit (AMD64)] ---------------+MODULES+------------------- Required: [X] numpy: 1.8.1 [X] scipy: 0.14.0 [X] pandas: 0.13.1 [X] matplotlib: 1.3.1 [X] statsmodels: 0.5.0 [X] sqlalchemy: 0.9.4 [X] traitsui: 4.4.0 [X] traits: 4.4.0 [X] enable: 4.3.0 [X] scikits.samplerate: present (no version) [X] obspy: 0.9.2 [X] sphinx: 1.2.2 [X] jinja2: 2.7.2 Backends: (at least one is required) [X] wx: 3.0.0.0 [X] PyQt4: present (no version) [X] PySide: 1.2.1 Not required, just checking: [X] setuptools: 3.6 [X] reportlab: $Id$ [X] configobj: 5.0.5 [X] pkg_resources: present (no version) [X] paramiko: 1.14.0 [X] ctypes: 1.1.0 [X] pyparsing: 2.0.1 [X] distutils: 2.7.6 [X] IPython: 2.1.0 [X] vtk: present (no version) 4. Are you using a MySQL server ? On your local machine or on a remote one ? Ans : I use local domain 127.0.0.1, MySQL server using WAMP or XAMPP (I tried both of this) in windows 8.1, and use MariaDB + httpd in CentOS. Thank you for your time Regards,, Dear Prima, First, (the easy answer), the fact that the GPU stays at 0% is perfectly normal, as MSNoise doesn't use GPU calculation (or Cuda, or anything related to GPUs). The increase between the -t 1 and -t 2 seems logical in terms of timing. But from 2 to 4, you don't seem to gain any timing increase. So: - How do you calculate/get the RAM/CPU values ? - What OS do you work on ? - Which Python version/architecture do you use ? (use bugreport.py -a and send it to the list) - Are you sure your machine is a quadcore ? or could it be a dual core,capable of hyperthreading (2x2) ? - Are you using a MySQL server ? On your local machine or on a remote one ? Scanning the archive actually doesn't consume much power, but requires quite a large number of readings, so your disk performance could be an issue too. I haven't yet played a lot with Numba, it's a thing to check in the future. If you succeed in doing that, PR (pull requests) are welcome ! This said, there are few chances the first steps (scan + jobs) can be accelerated dramatically, as the first mainly relies on Obspy and the second on the power of your database machine. Best, Thomas Le 17/05/2014 13:25, Wardhani, Prima Wira Kusuma a écrit : Dear Thomas, Dear All, I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise using CPU-GPU-RAM computing. I processed 49 seismic station, with 2 months data available. For scanning the archive *s01scan_archive.py*, using --init option, and thread option : --init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours --init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours --init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours the next threads option RAM comsume stuck in 1.6GB, no increase processing using multicore CPU. After finish I tried create new jobs using *s02new_jobs.py* for all station, and it takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the process only consume 1.6GB RAM, 17% CPU, no more than this value. How to optimalize the Processing using RAM-CPU-and GPU? Because it takes along time to process all my Data,, at least it can be optimize using RAM-CPU it will be better. My Computer using 16GB RAM and 4Core(a)3.2GHz, but it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason, it failed,. Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize MSNoise code? Thank you, WARDHANI, Prima Wira Kusuma Master of Earthquake Engineering and Engineering Seismology Universite Joseph Fourier - ISTerre Room 015prima.dhan...(a)gmail.com +62-81548815815 +33-663964113 がんばります！ _______________________________________________ MSNoise mailing listMSNoise@mailman-as.oma.behttp://mailman-as.oma.be/mailman/listinfo/msnoise

10 years, 11 months

Problem using 07.plot_dtt.py.

by Oscar Alberto Castro Artola

Hello, I am trying to run *07.plot_dtt.py <http://07.plot_dtt.py>* but there seems to be a problem: *oscar@bayta:~/sweet_noise$ py 07.plot_dtt.py <http://07.plot_dtt.py> * * /home/oscar/anaconda/lib/python2.7/site- packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9* * warnings.warn(_msg, ModuleDeprecationWarning)* * /home/oscar/anaconda/lib/python2.7/site-packages/setuptools-2.2- py2.7.egg/pkg_resources.py:991: UserWarning: /home/oscar/.python-eggs is writable by group/others and vulnerable to attack when used with get_resource_filename. Consider a more secure location (set with .set_extraction_path or the PYTHON_EGG_CACHE environment variable).* * loading 1 days* * Traceback (most recent call last):* * File "07.plot_dtt.py <http://07.plot_dtt.py>", line 115, in <module>* * py1_wmean, py1_wstd = get_wavgwstd(allbut)* * File "07.plot_dtt.py <http://07.plot_dtt.py>", line 59, in get_wavgwstd* * h = grouped.apply(wstd)* * File "/home/oscar/anaconda/lib/python2.7/site- packages/pandas/core/groupby.py", line 424, in apply* * return self._python_apply_general(f)* * File "/home/oscar/anaconda/lib/python2.7/site- packages/pandas/core/groupby.py", line 427, in _python_apply_general* * keys, values, mutated = self.grouper.apply(f, self.obj, self.axis)* * File "/home/oscar/anaconda/lib/python2.7/site- packages/pandas/core/groupby.py", line 883, in apply* * res = f(group)* * File "/home/oscar/anaconda/lib/python2.7/site- packages/pandas/core/groupby.py", line 422, in f* * return func(g, *args, **kwargs)* * File "07.plot_dtt.py <http://07.plot_dtt.py>", line 52, in wstd* * wstd = np.sqrt(np.sum(w * (d - wavg) ** 2) / ((N - 1) * np.sum(w) / N))* * ZeroDivisionError: float division by zero* I do not have much experiencie using python. if anyone has any idea I will appreciate it! Oscar

10 years, 11 months

s03compute_cc number of points issue

by Holly Godfrey

Dear All, I am having difficulty the with s03_compute_cc step. I keep getting the error Traceback (most recent call last): File "s03compute_cc.py", line 284, in <module> tramef_Z[istation] = trame ValueError: operands could not be broadcast together with shapes (1728000) (1727998) I am using input data in SAC format at a sampling rate of 20 Hz. I have checked and checked and checked and all files have 1728000 points in them. In MSNoise I have the resampling rate at 20 Hz, and have also tried it at 10 Hz (where I get ValueError: operands could not be broadcast together with shapes (864000) (863998)). I have tried many different jobs with different stations and days - all give this error. I have used version 1.2.3 without issues on 50 Hz input data (same data as I am using now, just with a lower sampling rate to increase processing speed and decrease file size) and resampling at 25 Hz, but both version 1.2.3 and 1.2.4 don't work for the 20 Hz. I don't understand where the two data points disappear to! Has anyone had this issue? Thanks, -- *Holly Godfrey* Institute of Geophysics, Victoria University of Wellington hollyjoannegodfrey(a)gmail.com

10 years, 11 months

MSNoise CPU-GPU-RAM Processing

by Wardhani, Prima Wira Kusuma

Dear Thomas, Dear All, I want to ask about Scan Archive, Creating New Jobs and CC using MSNoise using CPU-GPU-RAM computing. I processed 49 seismic station, with 2 months data available. For scanning the archive *s01scan_archive.py*, using --init option, and thread option : --init -t 1 : consume 400MB RAM, 17% CPU, 0% GPU : ~24 hours --init -t 2 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours --init -t 4 : consume 1.6GB RAM, 17% CPU, 0% GPU : ~13 hours the next threads option RAM comsume stuck in 1.6GB, no increase processing using multicore CPU. After finish I tried create new jobs using *s02new_jobs.py* for all station, and it takes 38 minutes for [DEBUG] all station pair for 1 day. I checked the process only consume 1.6GB RAM, 17% CPU, no more than this value. How to optimalize the Processing using RAM-CPU-and GPU? Because it takes along time to process all my Data,, at least it can be optimize using RAM-CPU it will be better. My Computer using 16GB RAM and 4Core(a)3.2GHz, but it only used 1.6GB only.. I tried to insert NumbaPro, but for some reason, it failed,. Can us suggest how to use parallel computing using GPU-CPU-RAM for optimize MSNoise code? Thank you, WARDHANI, Prima Wira Kusuma Master of Earthquake Engineering and Engineering Seismology Universite Joseph Fourier - ISTerre Room 015 prima.dhani06(a)gmail.com +62-81548815815 +33-663964113 がんばります！

10 years, 11 months

NameError: global name 'gps2DistAzimuth' is not defined

by Dimitris Giannopoulos

Dear mailing list, I am facing a problem concerning the cc computation step. Specifically, I have an error message about the coordinates of the stations. I am processing the data of two stations. I have already completed the previous steps without any problem. The following is the error that I have when I execute the "s03compute_cc.py" script: _________________________ Traceback (most recent call last): File "s03compute_cc.py", line 335, in <module> cplAz = azimuth(coordinates, X0, Y0, X1, Y1) File "/localstorage/MSNoise/database_tools.py", line 470, in azimuth dist, azim, bazim = gps2DistAzimuth(y0, x0, y1, x1) NameError: global name 'gps2DistAzimuth' is not defined _____________________________________________ I am going to process R and T components, so the coordinates of the stations are inputted in the station table as it is needed. I think that I did it properly. Did anybody face a similar problem? I also attach a screen-shot of the terminal in order to see when exactly the error appears. Thank you very much in advance. Dimitris

10 years, 11 months

← Newer
1
...
27
28
29
30
31
32
33
34
35
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

MSNoise