MSNoise June 2017

msnoise@lists.observatory.be

12 participants
8 discussions

by LUIS FRANCO MARIN

Dear Thomas et al., I installed MSNoise 1.5 again and started the workflow again. When I run "msnoise scan_archive --init" with MSNoise 1.5 and Python 1.7, the result by screen is: ---------------------------------------------------- luis@luis-franco:/media/luis/Storage/MSeed_Copahue$ msnoise scan_archive --init 2017-06-28 17:02:12 [INFO] *** Starting: Scan Archive *** 2017-06-28 17:02:12 [INFO] Initializing (should be run only once) 2017-06-28 17:02:12 [INFO] Will work on 1 thread(s) 2017-06-28 17:02:12 [INFO] Will search for files between 2012-01-01 and 2016-12-31 2017-06-28 17:02:12 [DEBUG] Will search for channels: [u'*'] 2017-06-28 17:02:12 [DEBUG] Data Folder: /media/luis/Storage/MSeed_Copahue 2017-06-28 17:02:12 [DEBUG] Folders to glob: /media/luis/Storage/MSeed_Copahue/2012/TC/BOSZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/COPZ/*.D , /media/luis/Storage/MSeed_Copahue/2012/TC/FR2Z/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/FROZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/HITZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/MAHZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/MAQZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/MLZZ/*.D, /media/luis/Storage/MSeed_Copahue/2012/TC/PEHZ/*.D, /media/luis/Storage/MSeed_Copa I installed MSNoise 1.5 again and started the workflow again hue/2012/TC/RALZ/*.D, /media/luis/Storage/MSeed_Copahue/2013/TC/BOSZ/*.D, /media/luis/Storage/MSeed_Copahue/2013/TC/COPZ/*.D, /media/luis/Storage/MSeed_Copahue/2013/TC/FR2Z/*.D, ... 2017-06-28 17:02:12 [INFO] *** Finished: Scan Archive *** 2017-06-28 17:02:12 [INFO] It took 0.02 seconds ---------------------------------------------------- My "msnoise info" is *********************************** General: MSNoise is installed in: /home/luis/anaconda2/lib/python2.7/site-packages - db.ini is present Configuration: - ./ exists - CROSS_CORRELATIONS does not exists (and that is normal because keep_all=False) Raw config bits: "D"efault or "M"odified (green) M data_folder: ./ D output_folder: CROSS_CORRELATIONS D data_structure: SDS D network: * D channels: * M startdate: 2012-01-01 M enddate: 2016-12-31 D analysis_duration: 86400 D cc_sampling_rate: 20.0 M resampling_method: Resample D preprocess_lowpass: 8.0 D preprocess_highpass: 0.01 D remove_response: N D response_format: dataless D response_path: inventory D response_prefilt: (0.005, 0.006, 30.0, 35.0) D maxlag: 120. D corr_duration: 1800. M overlap: 0.5 D windsorizing: 3 D whitening: A D stack_method: linear D pws_timegate: 10.0 D pws_power: 2.0 D crondays: -1 D components_to_compute: ZZ D autocorr: N D keep_all: N D keep_days: Y M ref_begin: 2012-01-01 M ref_end: 2016-12-31 D mov_stack: 5General: MSNoise is installed in: /home/luis/anaconda2/lib/python2.7/site-packages - db.ini is present Configuration: - ./ exists - CROSS_CORRELATIONS does not exists (and that is normal because keep_all=False) Raw config bits: "D"efault or "M"odified (green) M data_folder: ./ D output_folder: CROSS_CORRELATIONS D data_structure: SDS D network: * D channels: * M startdate: 2012-01-01 M enddate: 2016-12-31 D analysis_duration: 86400 D cc_sampling_rate: 20.0 M resampling_method: Resample D preprocess_lowpass: 8.0 D preprocess_highpass: 0.01 D remove_response: N D response_format: dataless D response_path: inventory D response_prefilt: (0.005, 0.006, 30.0, 35.0) D maxlag: 120. D corr_duration: 1800. M overlap: 0.5 D windsorizing: 3 D whitening: A D stack_method: linear D pws_timegate: 10.0 D pws_power: 2.0 D crondays: -1 D components_to_compute: ZZ D autocorr: N D keep_all: N D keep_days: Y M ref_begin: 2012-01-01 M ref_end: 2016-12-31 D mov_stack: 5 D export_format: MSEED D sac_format: doublets D dtt_lag: static D dtt_v: 1.0 D dtt_minlag: 5.0 D dtt_width: 30.0 D dtt_sides: both D dtt_mincoh: 0.65 D dtt_maxerr: 0.1 D dtt_maxdt: 0.1 D plugins: Filters: ID: [low:high] [mwcs_low:mwcs_high] mwcs_wlen mwcs_step used 01: [0.010:8.000] [0.020:0.850] 020 010 Y Stations: TC.BOSZ -71.0100 -37.8490 1660.0 DEG Y TC.COPZ -71.1600 -37.8370 2631.0 DEG Y TC.FR2Z -71.1210 -37.8200 2283.0 DEG Y TC.FROZ -71.1210 -37.8180 2283.0 DEG Y TC.HITZ -71.1130 -37.8870 2288.0 DEG Y TC.MAHZ -71.1690 -37.8890 2044.0 DEG Y TC.MAQZ -71.0730 -37.8360 2014.0 DEG Y TC.MLZZ -71.1160 -37.8430 2100.0 DEG Y TC.PEHZ -71.2030 -37.8520 2496.0 DEG Y TC.RALZ -71.2060 -37.8770 2313.0 DEG Y CC Jobs: DTT Jobs: D export_format: MSEED D sac_format: doublets D dtt_lag: static D dtt_v: 1.0 D dtt_minlag: 5.0 D dtt_width: 30.0 D dtt_sides: both D dtt_mincoh: 0.65 D dtt_maxerr: 0.1 D dtt_maxdt: 0.1 D plugins: Filters: ID: [low:high] [mwcs_low:mwcs_high] mwcs_wlen mwcs_step used 01: [0.010:8.000] [0.020:0.850] 020 010 Y Stations: TC.BOSZ -71.0100 -37.8490 1660.0 DEG Y TC.COPZ -71.1600 -37.8370 2631.0 DEG Y TC.FR2Z -71.1210 -37.8200 2283.0 DEG Y TC.FROZ -71.1210 -37.8180 2283.0 DEG Y TC.HITZ -71.1130 -37.8870 2288.0 DEG Y TC.MAHZ -71.1690 -37.8890 2044.0 DEG Y TC.MAQZ -71.0730 -37.8360 2014.0 DEG Y TC.MLZZ -71.1160 -37.8430 2100.0 DEG Y TC.PEHZ -71.2030 -37.8520 2496.0 DEG Y TC.RALZ -71.2060 -37.8770 2313.0 DEG Y CC Jobs: DTT Jobs: *********************************** If the seismic files in MSeed format are OK, if in other times the MSNoise did run correctely, I don't understand why "msnoise scan_archive" not find the MSeed files ? Thanks for your suggestions, Regards, Luis -- LUIS ENRIQUE FRANCO Volcano Observatory Chile

8 years

30 citations for MSNoise !

by Thomas Lecocq

Dear MSNoise fans, I'm very proud to announce that we just reached 30 citations for MSNoise !! I've updated the list on http://www.msnoise.org/they-cite-msnoise/ Congratulations to all authors for so many brilliant papers! And thanks for using and acknowledging MSNoise !! MSNoise is 30 Citations, among which * 6 in GRL * 4 in Science Advances * 2 in JGR * 3 in JVGR * 2 in SEG-related journals * 1 in SRL * 1 in GJI * etc... Best regards, Thomas -- Dr. Thomas Lecocq Geologist - Seismologist Seismology - Gravimetry Royal Observatory of Belgium * * * * * * * * * * --------- http://www.seismology.be http://msnoise.org http://twitter.com/#!/Seismologie_be https://www.facebook.com/seismologie.be

8 years

MSNoise: Decimation_factor

by LUIS FRANCO MARIN

Dear Thomas et al, When I run "msnoise compute_cc", this error appears on the screen: *********************************** Process Process-1: Traceback (most recent call last): File "/home/luis/anaconda2/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/home/luis/anaconda2/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "/home/luis/anaconda2/lib/python2.7/site-packages/msnoise/s03compute_cc.py", line 384, in main params.decimation_factor = int(get_config(db, "decimation_factor")) ValueError: invalid literal for int() with base 10: '' ************************************* This value does not appear to edit in "localhost: 5000" Is it possible to have another inconvenient with my MSNoise configuration ? ------------------------------------------------------------------ >msnoise info ------------------------------------------------------------------ General: MSNoise is installed in: /home/luis/anaconda2/lib/python2.7/site-packages - db.ini is present Configuration: - ./ exists - CROSS_CORRELATIONS does not exists (and that is normal because keep_all=False) Raw config bits: "D"efault or "M"odified (green) M data_folder: ./ D output_folder: CROSS_CORRELATIONS D data_structure: SDS D network: * D channels: * M startdate: 2014-01-01 M enddate: 2016-12-31 D analysis_duration: 86400 M cc_sampling_rate: 5 D resampling_method: Resample M decimation_factor: M preprocess_lowpass: 0.5 M preprocess_highpass: 0.025 D remove_response: N D response_format: dataless D response_path: inventory D response_prefilt: (0.005, 0.006, 30.0, 35.0) D maxlag: 120. D corr_duration: 1800. M overlap: 0.5 D windsorizing: 3 D stack_method: linear D pws_timegate: 10.0 D pws_power: 2.0 D crondays: -1 M ZZ: M ZR: M ZT: M RZ: M RR: M RT: M TZ: M TR: M TT: D autocorr: N D keep_all: N D keep_days: Y M ref_begin: 2014-01-01 M ref_end: 2016-12-31 M mov_stack: 1,2,5,10 D export_format: MSEED D sac_format: doublets D dtt_lag: static D dtt_v: 1.0 D dtt_minlag: 5.0 D dtt_width: 30.0 D dtt_sides: both D dtt_mincoh: 0.65 D dtt_maxerr: 0.1 D dtt_maxdt: 0.1 D plugins: Filters: ID: [low:high] [mwcs_low:mwcs_high] mwcs_wlen mwcs_step used 01: [0.025:0.500] [0.025:0.500] 010 005 Y Stations: TC.AD2Z 0.0000 0.0000 0.0 UTM Y TC.BARZ 0.0000 0.0000 0.0 UTM Y TC.CARZ 0.0000 0.0000 0.0 UTM Y TC.CR2Z 0.0000 0.0000 0.0 UTM Y TC.CR3Z 0.0000 0.0000 0.0 UTM Y TC.CRIZ 0.0000 0.0000 0.0 UTM Y TC.ICEZ -70.5030 -35.2510 2454.0 DEG Y TC.PETZ -70.5670 -35.3020 3397.0 DEG Y TC.TENZ 0.0000 0.0000 0.0 UTM Y TC.VIEZ -70.5790 -35.2100 3234.0 DEG Y CC Jobs: T : 4918 DTT Jobs: --------------------------------------------------------- Best regards, Luis -- LUIS ENRIQUE FRANCO Volcano Observatory Chile

8 years

Problem creating database.

by Oscar Alberto Castro Artola

Hello, I am migrating my projects to a new and faster Linux computer. Currently I am working with MSNoise 1.5 and everything works fine except for one thing: I have created a new database usgin phpmyadmin (browser interface) but when I run *msnoise install *I get the following error: Traceback (most recent call last): File "/usr/local/anaconda/bin/msnoise", line 11, in <module> sys.exit(run()) File "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/scripts/msnoise.py", line 717, in run cli(obj={}) File "/usr/local/anaconda/lib/python2.7/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/usr/local/anaconda/lib/python2.7/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/usr/local/anaconda/lib/python2.7/site-packages/click/core.py", line 1066, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/anaconda/lib/python2.7/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/anaconda/lib/python2.7/site-packages/click/core.py", line 535, in invoke return callback(*args, **kwargs) File "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/scripts/msnoise.py", line 211, in install main() File "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/s000installer.py", line 93, in main Base.metadata.create_all(engine) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/sql/schema.py", line 3934, in create_all tables=tables) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1928, in _run_visitor with self._optional_conn_ctx_manager(connection) as conn: File "/usr/local/anaconda/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1921, in _optional_conn_ctx_manager with self.contextual_connect() as conn: File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2112, in contextual_connect self._wrap_pool_connect(self.pool.connect, None), File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2151, in _wrap_pool_connect e, dialect, self) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1465, in _handle_dbapi_exception_noconnection exc_info File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2147, in _wrap_pool_connect return fn() File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect return _ConnectionFairy._checkout(self) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout rec = pool._do_get() File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get self._dec_overflow() File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__ compat.reraise(exc_type, exc_value, exc_tb) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get return self._create_connection() File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection return _ConnectionRecord(self) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__ self.__connect(first_connect_check=True) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/pool.py", line 651, in __connect connection = pool._invoke_creator(self) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 105, in connect return dialect.connect(*cargs, **cparams) File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 393, in connect return self.dbapi.connect(*cargs, **cparams) File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/__init__.py", line 90, in Connect return Connection(*args, **kwargs) File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 706, in __init__ self.connect() File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 932, in connect self._request_authentication() File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 1152, in _request_authentication auth_packet = self._read_packet() File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 1014, in _read_packet packet.check_error() File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 393, in check_error err.raise_mysql_exception(self._data) File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, u"Access denied for user 'oscar'@'localhost' (using password: YES)") The database created has my username on it with all privileges. I hope you can help me! Thank you! Oscar

8 years

Problem with database conection.

by Oscar Alberto Castro Artola

Hello, I have 10 years of data for 13 stations, I have computed the CCF and now it is turn to compute the MWCS. When I do *msnoise -t 20 compute_mwcs *this error appears: *Process Process-4:* *Traceback (most recent call last):* * File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap* * self.run()* * File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py", line 114, in run* * self._target(*self._args, **self._kwargs)* * File "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/s05compute_mwcs.py", line 150, in main* * update_job(db, day, pair, jobtype='DTT', flag='D')* * File "/usr/local/anaconda/lib/python2.7/site-packages/msnoise/api.py", line 615, in update_job* * job = session.query(Job).filter(Job.day == day).filter(Job.pair == pair).filter(Job.jobtype == jobtype).first()* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2755, in first* * ret = list(self[0:1])* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2547, in __getitem__* * return list(res)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2855, in __iter__* * return self._execute_and_instances(context)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2878, in _execute_and_instances* * result = conn.execute(querycontext.statement, self._params)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 945, in execute* * return meth(self, multiparams, params)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection* * return connection._execute_clauseelement(self, multiparams, params)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement* * compiled_sql, distilled_params* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1189, in _execute_context* * context)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception* * exc_info* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause* * reraise(type(exception), exception, tb=exc_tb, cause=cause)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context* * context)* * File "/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 470, in do_execute* * cursor.execute(statement, parameters)* * File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py", line 158, in execute* * result = self._query(query)* * File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py", line 308, in _query* * conn.query(q)* * File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 819, in query* * self._execute_command(COMMAND.COM_QUERY, sql)* * File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 1037, in _execute_command* * self._write_bytes(packet)* * File "/usr/local/anaconda/lib/python2.7/site-packages/pymysql/connections.py", line 989, in _write_bytes* * raise err.OperationalError(2006, "MySQL server has gone away (%r)" % (e,))* *OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (error(32, 'Broken pipe'))") [SQL: u'SELECT jobs.ref AS jobs_ref, jobs.day AS jobs_day, jobs.pair AS jobs_pair, jobs.jobtype AS jobs_jobtype, jobs.flag AS jobs_flag, jobs.lastmod AS jobs_lastmod \nFROM jobs \nWHERE jobs.day = %(day_1)s AND jobs.pair = %(pair_1)s AND jobs.jobtype = %(jobtype_1)s \n LIMIT %(param_1)s'] [parameters: {u'param_1': 1, u'day_1': '2009-02-16', u'jobtype_1': 'DTT', u'pair_1': 'IG.ARIG:IG.MOIG'}]* At the beginning everything works fine, but because it takes a long time to make the computation (around 5 days), every time that I come back to see the results this happens. I am working with MSNoise 1.4 and I don't know how to upgrade to MSNoise 1.5. Does anyone know how to fix this? Thanks in advance! Oscar

8 years

SAC header

by Crowder, Emily

I have set "maxlag" to be 300 s, so I would like cross-correlations centred on zero, +/- 300 s. Instead, MSNoise is giving me cross-correlations from 0 s to 600 s. In the SAC header B = 0.000000e+00, and E = 6.000000e+02. Why is this happening? How can it be corrected? Thanks in advance, Emily ~~~~~~~~~~~~~~~~~~~~~~~~~ Emily Crowder MSc BSc PhD Student School of Geosciences University of Aberdeen Aberdeen AB24 3UE Scotland Em: emily.crowder(a)abdn.ac.uk ~~~~~~~~~~~~~~~~~~~~~~~~~ The University of Aberdeen is a charity registered in Scotland, No SC013683. Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. SC013683.

8 years, 1 month

Is it possible to plot the correlation coefficients?

by aeroca＠insivumeh.gob.gt

Hello everyone, I was wonder if it is possible to plot the correlation coefficients using MSnoise. I know that I have to compute the CCF reference first and then using the moving-window stacks with a daily CCF I can get the correlation coefficients, but how do i get them plotted? I will appreciate some help! Thank you.

8 years, 1 month

archive scan crash

by Pálmi Erlendsson

Greetings! I was doing initial scan of my archive: msnoise -t 2 scan_archive --init This is a SDS structured archive and msnoise stumbled upon a shell script, tmp.sh, in the wrong place after successfully scaning 273271 files: [msnoise@gaur SUH]$ pwd /mnt/archive/2016/IM/SUH [msnoise@gaur SUH]$ ls HHE.D HHN.D HHZ.D tmp.sh [msnoise@gaur SUH]$ The scan crashed and complained that tmp.sh was not a direcory. Shouldn't msnoise just ignore files of wrong format? I am running msnoise 1.5 on CentOS Linux release 7.2.1511 machine ... if that is of any interest :) cheers palmi Pálmi Erlendsson Jarðfræðingur Sími / Tel: +354 528 1536 gsm: +354 841 2938 ÍSOR / Iceland GeoSurvey Grensásvegur 9, 108 Reykjavík - www.isor.is ___________________________________

8 years, 1 month

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

MSNoise June 2017