MSNoise

msnoise@lists.observatory.be

349 discussions

by Thomas Lecocq

Dear all, I've tagged a new release on github. From the release notes: /Thanks to some great early MSNoise adopters who reported problems using the mailing-list, we have identified a few bugs and tricky situations where some steps failed. This is the case for/ / / * /scan_archive: there were major issues using Threads, this step uses Process for multiprocessing, which is much safer. The only problem remaining is that there is no more console-logging of the found/identified files. To be corrected soon./ * /new_jobs: mostly rewritten, all jobs are properly identified now. There should no more lost jobs !/ /I've also reflected those changes in the documentation. / To update your MSNoise install, just download the new .py files and replace them. There is no compatibility issue, as the database schema has not changed. Thanks a lot, again, to all of you for your feedback (in- or off-list), it helps making MSNoise more robust every day ! Cheers, Thomas //

11 years

new jobs not registering

by Kasper van Wijk

Dear MsNoise-ers: I managed to run the CC's on the EH* components on 11 stations with 3 days (Julian days 102,103,104) worth of data initialized with python s01scan_archive.py -i -t 1 New jobs were created, and life was good. Then, I added more days (days 105-106): [kasper@localhost ABAZ]$ ls ABAZ.NZ..EHE.2014.102 ABAZ.NZ..EHZ.2014.106 ABAZ.NZ..VEM.2014.105 ABAZ.NZ..EHE.2014.103 ABAZ.NZ..LCQ.2014.102 ABAZ.NZ..VEM.2014.106 ABAZ.NZ..EHE.2014.104 ABAZ.NZ..LCQ.2014.103 ABAZ.NZ..VEP.2014.102 ABAZ.NZ..EHE.2014.105 ABAZ.NZ..LCQ.2014.104 ABAZ.NZ..VEP.2014.103 ABAZ.NZ..EHE.2014.106 ABAZ.NZ..LCQ.2014.105 ABAZ.NZ..VEP.2014.104 ABAZ.NZ..EHN.2014.102 ABAZ.NZ..LCQ.2014.106 ABAZ.NZ..VEP.2014.105 ABAZ.NZ..EHN.2014.103 ABAZ.NZ..VEC.2014.102 ABAZ.NZ..VEP.2014.106 ABAZ.NZ..EHN.2014.104 ABAZ.NZ..VEC.2014.103 ABAZ.NZ..VKI.2014.102 ABAZ.NZ..EHN.2014.105 ABAZ.NZ..VEC.2014.104 ABAZ.NZ..VKI.2014.103 ABAZ.NZ..EHN.2014.106 ABAZ.NZ..VEC.2014.105 ABAZ.NZ..VKI.2014.104 ABAZ.NZ..EHZ.2014.102 ABAZ.NZ..VEC.2014.106 ABAZ.NZ..VKI.2014.105 ABAZ.NZ..EHZ.2014.103 ABAZ.NZ..VEM.2014.102 ABAZ.NZ..VKI.2014.106 ABAZ.NZ..EHZ.2014.104 ABAZ.NZ..VEM.2014.103ABAZ.NZ..EHZ.2014.105 ABAZ.NZ..VEM.2014.104 and ran: python s01scan_archive.py -t 1 Which I thought should flag the data base that there is work to do, but no new jobs are created! In fact, the data is not added to the data base (see attached screenshot), and hence there is nothing to CC. Any ideas to try would be appreciated. Best, kasper

11 years

Re: [MSNoise] (BUD) structure?

by Kasper van Wijk

Hi Thom: The errors happen both as -i (needs a fix in the documentation: no longer "init") and in chron. Here is an excerpt of the output with the print statement added as you suggested: [kasper@kvan637-277769 MSNoise-master]$ python -i s01scan_archive.py 2014-04-01 13:18:11,043 [INFO] *** Starting: Scan Archive *** 2014-04-01 13:18:11,130 [INFO] Will work on 1 threads ['', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHE.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHE.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHN.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHN.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHZ.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..EHZ.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..LCQ.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..LCQ.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEC.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEC.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEM.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEM.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEP.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VEP.2014.090', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VKI.2014.089', '/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/ABAZ.NZ..VKI.2014.090'] 2014-04-01 13:18:11,252 [INFO] Started: /home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ 2014-04-01 13:18:11,276 [INFO] child process calling self.run() Problem [Errno 21] Is a directory: u'/home/kasper/Downloads/MSNoise-master/AUCK/NZ/ABAZ/' 2014-04-01 13:18:13,215 [INFO] process shutting down 2014-04-01 13:18:13,215 [INFO] process exiting with exitcode 0 And maybe it is not too bad, but I don't know enough about the sqlite to understand if the attached screenshot means the data_availability table is filled or not, but [kasper@kvan637-277769 MSNoise-master]$ python s02new_jobs.py 2014-04-01 13:21:01,630 [INFO] *** Starting: New Jobs *** 2014-04-01 13:21:01,664 [DEBUG] Found 0 new jobs to do 2014-04-01 13:21:01,839 [INFO] *** Finished: New Jobs *** there are no new jobs.. Cheers, Kasper On 31 March 2014 21:04, Thomas Lecocq <thomas.lecocq(a)oma.be> wrote: > Kasper, > > Does this happen when initialising the scan ? (- i argument) ? or in cron ? > Is the data_availability table filled, even with all the errors ? > > It looks like, when listing files in a directory, the "." or "/" line > gets returned, which leads to the read-error... This said, the ExitCode > 0 means it sucessfully finished the process. > > one thing to test: > > on Line 179, add : > > print files > > > with the same indentation as the following line (L180: if len(files) != > 0:). > > Best regards > > Thom > > > Le 31/03/2014 02:52, Kasper van Wijk a écrit : > > Dear Thomas, > > > > python s01scan_archive.py leads to many errors of this type: > > > > 2014-03-31 13:39:36,535 [INFO] Started: > > /home/kasper/Downloads/MSNoise-master/AUCK/NZ/WTAZ > > 2014-03-31 13:39:36,538 [INFO] child process calling self.run() > > Problem [Errno 21] Is a directory: > > u'/home/kasper/Downloads/MSNoise-master/AUCK/NZ/WTAZ/' > > 2014-03-31 13:39:36,603 [INFO] process shutting down > > 2014-03-31 13:39:36,603 [INFO] process exiting with exitcode 0 > > > > My data is in the following BUD structure: > > [kasper@kvan637-277769 AWAZ]$ pwd > > > > /home/kasper/Downloads/MSNoise-master/AUCK/NZ/ > > > > [kasper@kvan637-277769 NZ]$ ls -ltr > > total 44 > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 RVAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 AWAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 WIAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 MKAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 MBAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 HBAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 ETAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 ABAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 WTAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 KBAZ > > drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 EPAZ > > [kasper@kvan637-277769 NZ]$ cd AWAZ/ > > [kasper@kvan637-277769 AWAZ]$ ls > > AWAZ.NZ..EH1.2014.078 AWAZ.NZ..LCQ.2014.078 AWAZ.NZ..VEP.2014.078 > > AWAZ.NZ..EH1.2014.089 AWAZ.NZ..LCQ.2014.089 AWAZ.NZ..VEP.2014.089 > > AWAZ.NZ..EH2.2014.078 AWAZ.NZ..VEC.2014.078 AWAZ.NZ..VKI.2014.078 > > AWAZ.NZ..EH2.2014.089 AWAZ.NZ..VEC.2014.089 AWAZ.NZ..VKI.2014.089 > > AWAZ.NZ..EHZ.2014.078 AWAZ.NZ..VEM.2014.078 > > AWAZ.NZ..EHZ.2014.089 AWAZ.NZ..VEM.2014.089 > > > > I am only intending to xcorrelate the EHZ (and EHN and EHE, if they > exist). > > > > Config screen attached. > > > > Cheers, > > kasper > > > > > > _______________________________________________ > > MSNoise mailing list > > MSNoise(a)mailman-as.oma.be > > http://mailman-as.oma.be/mailman/listinfo/msnoise > > _______________________________________________ > MSNoise mailing list > MSNoise(a)mailman-as.oma.be > http://mailman-as.oma.be/mailman/listinfo/msnoise >

11 years, 1 month

(BUD) structure?

by Kasper van Wijk

Dear Thomas, python s01scan_archive.py leads to many errors of this type: 2014-03-31 13:39:36,535 [INFO] Started: /home/kasper/Downloads/MSNoise-master/AUCK/NZ/WTAZ 2014-03-31 13:39:36,538 [INFO] child process calling self.run() Problem [Errno 21] Is a directory: u'/home/kasper/Downloads/MSNoise-master/AUCK/NZ/WTAZ/' 2014-03-31 13:39:36,603 [INFO] process shutting down 2014-03-31 13:39:36,603 [INFO] process exiting with exitcode 0 My data is in the following BUD structure: [kasper@kvan637-277769 AWAZ]$ pwd /home/kasper/Downloads/MSNoise-master/AUCK/NZ/ [kasper@kvan637-277769 NZ]$ ls -ltr total 44 drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 RVAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 AWAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 WIAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 MKAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 MBAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 HBAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 ETAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 ABAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 WTAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 KBAZ drwxrwxr-x. 2 kasper kasper 4096 Mar 31 13:00 EPAZ [kasper@kvan637-277769 NZ]$ cd AWAZ/ [kasper@kvan637-277769 AWAZ]$ ls AWAZ.NZ..EH1.2014.078 AWAZ.NZ..LCQ.2014.078 AWAZ.NZ..VEP.2014.078 AWAZ.NZ..EH1.2014.089 AWAZ.NZ..LCQ.2014.089 AWAZ.NZ..VEP.2014.089 AWAZ.NZ..EH2.2014.078 AWAZ.NZ..VEC.2014.078 AWAZ.NZ..VKI.2014.078 AWAZ.NZ..EH2.2014.089 AWAZ.NZ..VEC.2014.089 AWAZ.NZ..VKI.2014.089 AWAZ.NZ..EHZ.2014.078 AWAZ.NZ..VEM.2014.078 AWAZ.NZ..EHZ.2014.089 AWAZ.NZ..VEM.2014.089 I am only intending to xcorrelate the EHZ (and EHN and EHE, if they exist). Config screen attached. Cheers, kasper

11 years, 1 month

(no subject)

by Oscar Alberto Castro Artola

Hi, I was trying tu run *s05compute_mwcs.py* but I get this error message: *Traceback (most recent call last):* * File "s05compute_mwcs.py", line 118, in <module>* * cur, ref, f.mwcs_low, f.mwcs_high, goal_sampling_rate, -maxlag, f.mwcs_wlen, f.mwcs_step)* * File "/home/oscar/MSNoise-1.2.3/MWCS.py", line 145, in mwcs* * res = sm.regression.linear_model.WLS(phi, v, w**2).fit()* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 381, in __init__* * weights=weights, hasconst=hasconst)* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 79, in __init__* * super(RegressionModel, self).__init__(endog, exog, **kwargs)* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py", line 136, in __init__* * super(LikelihoodModel, self).__init__(endog, exog, **kwargs)* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py", line 52, in __init__* * self.data = handle_data(endog, exog, missing, hasconst, **kwargs)* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 397, in handle_data* * return klass(endog, exog=exog, missing=missing, hasconst=hasconst, **kwargs)* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 78, in __init__* * self._check_integrity()* * File "/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 246, in _check_integrity* * if len(self.exog) != len(self.endog):* *TypeError: len() of unsized object* Thank you in advance! Oscar

11 years, 1 month

Re: [MSNoise] MSNoise Digest, Vol 5, Issue 6

by Esteban Chaves

Hi Thomas, I switch from sqlite to mysql and now the s02new_jobs.py works well. But I got this really weird error when I run: Estebans-MacBook-Pro:MSN ech$ python s05compute_mwcs.py /Users/ech/anaconda/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11: ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9 warnings.warn(_msg, ModuleDeprecationWarning) 2014-03-20 19:57:52,503 [INFO] *** Starting: Compute MWCS *** 2014-03-20 19:57:52,599 [INFO] We will recompute all MWCS based on the new REF for YZ.HRIU:YZ.INDI 2014-03-20 19:57:52,615 [INFO] We will recompute all MWCS based on the new REF for YZ.HRIU:YZ.NARJ 2014-03-20 19:57:52,631 [INFO] We will recompute all MWCS based on the new REF for YZ.INDI:YZ.NARJ 2014-03-20 19:57:52,714 [INFO] There are MWCS jobs for some days to recompute for YZ.HRIU:YZ.INDI 2014-03-20 19:57:52,721 [DEBUG] Processing MWCS for: YZ_HRIU_YZ_INDI.ZZ.01 - 2012-01-01 - 05 days Traceback (most recent call last): File "s05compute_mwcs.py", line 118, in <module> cur, ref, f.mwcs_low, f.mwcs_high, goal_sampling_rate, -maxlag, f.mwcs_wlen, f.mwcs_step) File "/Users/ech/sismo/MSN/MWCS.py", line 144, in mwcs res = sm.regression.linear_model.WLS(phi, v, w**2).fit() File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 381, in __init__ weights=weights, hasconst=hasconst) File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py", line 79, in __init__ super(RegressionModel, self).__init__(endog, exog, **kwargs) File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py", line 136, in __init__ super(LikelihoodModel, self).__init__(endog, exog, **kwargs) File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py", line 52, in __init__ self.data = handle_data(endog, exog, missing, hasconst, **kwargs) File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 397, in handle_data return klass(endog, exog=exog, missing=missing, hasconst=hasconst, **kwargs) File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 78, in __init__ self._check_integrity() File "/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line 246, in _check_integrity if len(self.exog) != len(self.endog): TypeError: len() of unsized object Thanks in advance, Esteban On Mar 14, 2014, at 5:00 AM, msnoise-request(a)mailman-as.oma.be wrote: > Send MSNoise mailing list submissions to > msnoise(a)mailman-as.oma.be > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman-as.oma.be/mailman/listinfo/msnoise > or, via email, send a message with subject or body 'help' to > msnoise-request(a)mailman-as.oma.be > > You can reach the person managing the list at > msnoise-owner(a)mailman-as.oma.be > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of MSNoise digest..." > > > Today's Topics: > > 1. Re: Found 0 new Jobs to do (Thomas Lecocq) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 13 Mar 2014 13:37:51 +0100 > From: Thomas Lecocq <thomas.lecocq(a)oma.be> > To: msnoise(a)mailman-as.oma.be > Subject: Re: [MSNoise] Found 0 new Jobs to do > Message-ID: <5321A69F.5090303(a)oma.be> > Content-Type: text/plain; charset=windows-1252; format=flowed > > Hi Esteban, > > If you use sqlite, you cannot use more than 1 thread, otherwise there > are big chances some data will be lost. So option "-t 1" is compulsory. > > With the new scan_archive code, there is no longer visual ouptut of the > scans, that is a problem I'm trying to solve soon. > > When you run the scan_archive -i -t1, it looks like your archive takes > some time to be scanned (72s), so I assume the scan is OK. Can you > confirm that after this step, the data_availability table contains > metadata for the station/files ? > > Then, what could happen is that you haven't properly configured the > start and end-times in the configurator, so no new jobs are found. > > > > Thomas > > Le 12/03/2014 20:35, Esteban Chaves a ?crit : >> Hi Tomas et al; >> >> I got the next problem: >> >> Estebans-MacBook-Pro:MSNoise-master ech$ python s002populate_station_table.py >> >>>> Populating the Station table >> Adding: YZ INDI >> Adding: YZ JUDI >> Adding: YZ SAJU >> Estebans-MacBook-Pro:MSNoise-master ech$ python s01scan_archive.py -i -t 2 >> 2014-03-12 12:14:23,805 [INFO] *** Starting: Scan Archive *** >> 2014-03-12 12:14:23,816 [INFO] Initializing (should be run only once) >> 2014-03-12 12:14:23,816 [INFO] Will work on 2 threads >> 2014-03-12 12:14:23,830 [INFO] Started: /Users/ech/gap/YZ/INDI >> 2014-03-12 12:14:23,833 [INFO] Started: /Users/ech/gap/YZ/INDI >> 2014-03-12 12:14:23,834 [INFO] child process calling self.run() >> 2014-03-12 12:14:23,836 [INFO] child process calling self.run() >> 2014-03-12 12:14:41,157 [INFO] process shutting down >> 2014-03-12 12:14:41,157 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:14:41,159 [INFO] process shutting down >> 2014-03-12 12:14:41,159 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:14:41,196 [INFO] Started: /Users/ech/gap/YZ/INDI >> 2014-03-12 12:14:41,200 [INFO] child process calling self.run() >> 2014-03-12 12:14:41,200 [INFO] Started: /Users/ech/gap/YZ/JUDI >> 2014-03-12 12:14:41,204 [INFO] child process calling self.run() >> 2014-03-12 12:14:56,235 [INFO] process shutting down >> 2014-03-12 12:14:56,236 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:14:56,275 [INFO] Started: /Users/ech/gap/YZ/JUDI >> 2014-03-12 12:14:56,279 [INFO] child process calling self.run() >> 2014-03-12 12:14:57,349 [INFO] process shutting down >> 2014-03-12 12:14:57,349 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:14:57,372 [INFO] Started: /Users/ech/gap/YZ/JUDI >> 2014-03-12 12:14:57,375 [INFO] child process calling self.run() >> 2014-03-12 12:15:09,552 [INFO] process shutting down >> 2014-03-12 12:15:09,552 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:15:09,578 [INFO] Started: /Users/ech/gap/YZ/SAJU >> 2014-03-12 12:15:09,581 [INFO] child process calling self.run() >> 2014-03-12 12:15:10,519 [INFO] process shutting down >> 2014-03-12 12:15:10,519 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:15:10,539 [INFO] Started: /Users/ech/gap/YZ/SAJU >> 2014-03-12 12:15:10,542 [INFO] child process calling self.run() >> 2014-03-12 12:15:24,492 [INFO] process shutting down >> 2014-03-12 12:15:24,492 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:15:24,493 [INFO] process shutting down >> 2014-03-12 12:15:24,494 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:15:24,519 [INFO] Started: /Users/ech/gap/YZ/SAJU >> 2014-03-12 12:15:24,522 [INFO] child process calling self.run() >> 2014-03-12 12:15:36,690 [INFO] process shutting down >> 2014-03-12 12:15:36,691 [INFO] process exiting with exitcode 0 >> 2014-03-12 12:15:36,709 [INFO] *** Finished: Scan Archive *** >> 2014-03-12 12:15:36,710 [INFO] It took 72.91 seconds >> 2014-03-12 12:15:36,710 [INFO] process shutting down >> Estebans-MacBook-Pro:MSNoise-master ech$ python s02new_jobs.py >> 2014-03-12 12:15:53,224 [INFO] *** Starting: New Jobs *** >> 2014-03-12 12:15:54,548 [DEBUG] Found 0 new jobs to do >> 2014-03-12 12:15:54,664 [INFO] *** Finished: New Jobs *** >> >> >> My data is in BUD format and I?m using the last MASTER release of MSNoise. This is the first time that I got this kind of problem. If you see my msnoise.sqlite table >> you will see that I?m trying to process data starting from day 2012-286 and basically (I think) it cannot initialize the jobs after scan. The weird thing is that using the previous days, say from May 2012 till September 2012 the programs works very well. >> >> I appreciate your help; >> >> Esteban >> PhD Student in Seismology >> University of California, Santa Cruz >> >> >> >> >> >> >> >> On Mar 9, 2014, at 5:00 AM, msnoise-request(a)mailman-as.oma.be wrote: >> >>> Send MSNoise mailing list submissions to >>> msnoise(a)mailman-as.oma.be >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> http://mailman-as.oma.be/mailman/listinfo/msnoise >>> or, via email, send a message with subject or body 'help' to >>> msnoise-request(a)mailman-as.oma.be >>> >>> You can reach the person managing the list at >>> msnoise-owner(a)mailman-as.oma.be >>> >>> When replying, please edit your Subject line so it is more specific >>> than "Re: Contents of MSNoise digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Fwd: MSNoise: filters (Thomas Lecocq) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Sat, 08 Mar 2014 16:56:15 +0100 >>> From: Thomas Lecocq <thomas.lecocq(a)seismology.be> >>> To: Python Package for Monitoring Seismic Velocity Changes using >>> Ambient Seismic Noise <msnoise(a)mailman-as.oma.be> >>> Subject: [MSNoise] Fwd: MSNoise: filters >>> Message-ID: <531B3D9F.3040003(a)seismology.be> >>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed >>> >>> Hi H?l?ne, >>> >>> The filters need to be defined in the configurator. Filter bounds define >>> the bandpass that will whitened. Around this whitened part, the spectra >>> will be tapered to 0.+0j. >>> >>> Indeed, I've rechecked the doc, it's not very obvious. Thanks for >>> pointing this out :) I'll try to spend some time on the doc soon. >>> >>> Briefly: each filter is defined with: >>> low : lower frequency of the bandpass >>> high : upper ... >>> >>> mwcs_low : slightly higher than low, it is the lower frequency for which >>> MWCS' weighted linear regression will be estimated >>> mwcs_high : slightly lower than high,... >>> >>> mwcs_wlen : window length for computing the relative delay in the MWCS >>> mwcs_step : step between two windows. if equal to mwcs_wlen: no overlap. >>> >>> rms_threshold: not used any more, should disappear, set it to 0.0. >>> >>> Hope this helps, let me know (via this mailing list, svp.) >>> >>> Thom >>> >>> -------- Message original -------- >>> Sujet: MSNoise: filters >>> Date : Sat, 8 Mar 2014 14:14:33 +0100 >>> >>> >>> Pour : Thomas Lecocq <thomas.lecocq(a)oma.be> >>> >>> >>> >>> Thomas, we solved the problem with scikits_samplerate apprently related to >>> some export in my .bashrc. >>> I am now at s03compute_cc.py and get an error because I did not define any >>> filters: apparently this is not implementted yet (?) , to what these filters >>> correspond? >>> Cheers >>> Helene >>> >>> -----Message d'origine----- >>> From: Thomas Lecocq >>> Sent: Thursday, March 06, 2014 5:45 PM >>> To: Helene.Lyon-Caen(a)ens.fr >>> Subject: Re: scikits >>> >>> H?l?ne, >>> >>> You could remove the dependency to scikits by using only Decimation, if >>> all your data are in the same sampling rate ! >>> >>> Thom >>> >>> Le 06/03/2014 16:18, Helene Lyon-Caen a ?crit : >>>> Thomas, >>>> we have been trying to understand with the computer engineer here. The >>>> only differences with what is on your web site we could find is that we >>>> use a more recent version of anaconda (1.9) and of python (2.7.6) >>>> could that make a difference? >>> >>> >>> >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> MSNoise mailing list >>> MSNoise(a)mailman-as.oma.be >>> http://mailman-as.oma.be/mailman/listinfo/msnoise >>> >>> >>> End of MSNoise Digest, Vol 5, Issue 4 >>> ************************************* >> _______________________________________________ >> MSNoise mailing list >> MSNoise(a)mailman-as.oma.be >> http://mailman-as.oma.be/mailman/listinfo/msnoise > > > > ------------------------------ > > _______________________________________________ > MSNoise mailing list > MSNoise(a)mailman-as.oma.be > http://mailman-as.oma.be/mailman/listinfo/msnoise > > > End of MSNoise Digest, Vol 5, Issue 6 > *************************************

11 years, 1 month

Found 0 new Jobs to do

by Esteban Chaves

Hi Tomas et al; I got the next problem: Estebans-MacBook-Pro:MSNoise-master ech$ python s002populate_station_table.py >> Populating the Station table Adding: YZ INDI Adding: YZ JUDI Adding: YZ SAJU Estebans-MacBook-Pro:MSNoise-master ech$ python s01scan_archive.py -i -t 2 2014-03-12 12:14:23,805 [INFO] *** Starting: Scan Archive *** 2014-03-12 12:14:23,816 [INFO] Initializing (should be run only once) 2014-03-12 12:14:23,816 [INFO] Will work on 2 threads 2014-03-12 12:14:23,830 [INFO] Started: /Users/ech/gap/YZ/INDI 2014-03-12 12:14:23,833 [INFO] Started: /Users/ech/gap/YZ/INDI 2014-03-12 12:14:23,834 [INFO] child process calling self.run() 2014-03-12 12:14:23,836 [INFO] child process calling self.run() 2014-03-12 12:14:41,157 [INFO] process shutting down 2014-03-12 12:14:41,157 [INFO] process exiting with exitcode 0 2014-03-12 12:14:41,159 [INFO] process shutting down 2014-03-12 12:14:41,159 [INFO] process exiting with exitcode 0 2014-03-12 12:14:41,196 [INFO] Started: /Users/ech/gap/YZ/INDI 2014-03-12 12:14:41,200 [INFO] child process calling self.run() 2014-03-12 12:14:41,200 [INFO] Started: /Users/ech/gap/YZ/JUDI 2014-03-12 12:14:41,204 [INFO] child process calling self.run() 2014-03-12 12:14:56,235 [INFO] process shutting down 2014-03-12 12:14:56,236 [INFO] process exiting with exitcode 0 2014-03-12 12:14:56,275 [INFO] Started: /Users/ech/gap/YZ/JUDI 2014-03-12 12:14:56,279 [INFO] child process calling self.run() 2014-03-12 12:14:57,349 [INFO] process shutting down 2014-03-12 12:14:57,349 [INFO] process exiting with exitcode 0 2014-03-12 12:14:57,372 [INFO] Started: /Users/ech/gap/YZ/JUDI 2014-03-12 12:14:57,375 [INFO] child process calling self.run() 2014-03-12 12:15:09,552 [INFO] process shutting down 2014-03-12 12:15:09,552 [INFO] process exiting with exitcode 0 2014-03-12 12:15:09,578 [INFO] Started: /Users/ech/gap/YZ/SAJU 2014-03-12 12:15:09,581 [INFO] child process calling self.run() 2014-03-12 12:15:10,519 [INFO] process shutting down 2014-03-12 12:15:10,519 [INFO] process exiting with exitcode 0 2014-03-12 12:15:10,539 [INFO] Started: /Users/ech/gap/YZ/SAJU 2014-03-12 12:15:10,542 [INFO] child process calling self.run() 2014-03-12 12:15:24,492 [INFO] process shutting down 2014-03-12 12:15:24,492 [INFO] process exiting with exitcode 0 2014-03-12 12:15:24,493 [INFO] process shutting down 2014-03-12 12:15:24,494 [INFO] process exiting with exitcode 0 2014-03-12 12:15:24,519 [INFO] Started: /Users/ech/gap/YZ/SAJU 2014-03-12 12:15:24,522 [INFO] child process calling self.run() 2014-03-12 12:15:36,690 [INFO] process shutting down 2014-03-12 12:15:36,691 [INFO] process exiting with exitcode 0 2014-03-12 12:15:36,709 [INFO] *** Finished: Scan Archive *** 2014-03-12 12:15:36,710 [INFO] It took 72.91 seconds 2014-03-12 12:15:36,710 [INFO] process shutting down Estebans-MacBook-Pro:MSNoise-master ech$ python s02new_jobs.py 2014-03-12 12:15:53,224 [INFO] *** Starting: New Jobs *** 2014-03-12 12:15:54,548 [DEBUG] Found 0 new jobs to do 2014-03-12 12:15:54,664 [INFO] *** Finished: New Jobs *** My data is in BUD format and I’m using the last MASTER release of MSNoise. This is the first time that I got this kind of problem. If you see my msnoise.sqlite table you will see that I’m trying to process data starting from day 2012-286 and basically (I think) it cannot initialize the jobs after scan. The weird thing is that using the previous days, say from May 2012 till September 2012 the programs works very well. I appreciate your help; Esteban PhD Student in Seismology University of California, Santa Cruz On Mar 9, 2014, at 5:00 AM, msnoise-request(a)mailman-as.oma.be wrote: > Send MSNoise mailing list submissions to > msnoise(a)mailman-as.oma.be > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman-as.oma.be/mailman/listinfo/msnoise > or, via email, send a message with subject or body 'help' to > msnoise-request(a)mailman-as.oma.be > > You can reach the person managing the list at > msnoise-owner(a)mailman-as.oma.be > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of MSNoise digest..." > > > Today's Topics: > > 1. Fwd: MSNoise: filters (Thomas Lecocq) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 08 Mar 2014 16:56:15 +0100 > From: Thomas Lecocq <thomas.lecocq(a)seismology.be> > To: Python Package for Monitoring Seismic Velocity Changes using > Ambient Seismic Noise <msnoise(a)mailman-as.oma.be> > Subject: [MSNoise] Fwd: MSNoise: filters > Message-ID: <531B3D9F.3040003(a)seismology.be> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi H?l?ne, > > The filters need to be defined in the configurator. Filter bounds define > the bandpass that will whitened. Around this whitened part, the spectra > will be tapered to 0.+0j. > > Indeed, I've rechecked the doc, it's not very obvious. Thanks for > pointing this out :) I'll try to spend some time on the doc soon. > > Briefly: each filter is defined with: > low : lower frequency of the bandpass > high : upper ... > > mwcs_low : slightly higher than low, it is the lower frequency for which > MWCS' weighted linear regression will be estimated > mwcs_high : slightly lower than high,... > > mwcs_wlen : window length for computing the relative delay in the MWCS > mwcs_step : step between two windows. if equal to mwcs_wlen: no overlap. > > rms_threshold: not used any more, should disappear, set it to 0.0. > > Hope this helps, let me know (via this mailing list, svp.) > > Thom > > -------- Message original -------- > Sujet: MSNoise: filters > Date : Sat, 8 Mar 2014 14:14:33 +0100 > > > Pour : Thomas Lecocq <thomas.lecocq(a)oma.be> > > > > Thomas, we solved the problem with scikits_samplerate apprently related to > some export in my .bashrc. > I am now at s03compute_cc.py and get an error because I did not define any > filters: apparently this is not implementted yet (?) , to what these filters > correspond? > Cheers > Helene > > -----Message d'origine----- > From: Thomas Lecocq > Sent: Thursday, March 06, 2014 5:45 PM > To: Helene.Lyon-Caen(a)ens.fr > Subject: Re: scikits > > H?l?ne, > > You could remove the dependency to scikits by using only Decimation, if > all your data are in the same sampling rate ! > > Thom > > Le 06/03/2014 16:18, Helene Lyon-Caen a ?crit : >> Thomas, >> we have been trying to understand with the computer engineer here. The >> only differences with what is on your web site we could find is that we >> use a more recent version of anaconda (1.9) and of python (2.7.6) >> could that make a difference? > > > > > > ------------------------------ > > _______________________________________________ > MSNoise mailing list > MSNoise(a)mailman-as.oma.be > http://mailman-as.oma.be/mailman/listinfo/msnoise > > > End of MSNoise Digest, Vol 5, Issue 4 > *************************************

11 years, 1 month

Fwd: MSNoise: filters

by Thomas Lecocq

Hi Hélène, The filters need to be defined in the configurator. Filter bounds define the bandpass that will whitened. Around this whitened part, the spectra will be tapered to 0.+0j. Indeed, I've rechecked the doc, it's not very obvious. Thanks for pointing this out :) I'll try to spend some time on the doc soon. Briefly: each filter is defined with: low : lower frequency of the bandpass high : upper ... mwcs_low : slightly higher than low, it is the lower frequency for which MWCS' weighted linear regression will be estimated mwcs_high : slightly lower than high,... mwcs_wlen : window length for computing the relative delay in the MWCS mwcs_step : step between two windows. if equal to mwcs_wlen: no overlap. rms_threshold: not used any more, should disappear, set it to 0.0. Hope this helps, let me know (via this mailing list, svp.) Thom -------- Message original -------- Sujet: MSNoise: filters Date : Sat, 8 Mar 2014 14:14:33 +0100 Pour : Thomas Lecocq <thomas.lecocq(a)oma.be> Thomas, we solved the problem with scikits_samplerate apprently related to some export in my .bashrc. I am now at s03compute_cc.py and get an error because I did not define any filters: apparently this is not implementted yet (?) , to what these filters correspond? Cheers Helene -----Message d'origine----- From: Thomas Lecocq Sent: Thursday, March 06, 2014 5:45 PM To: Helene.Lyon-Caen(a)ens.fr Subject: Re: scikits Hélène, You could remove the dependency to scikits by using only Decimation, if all your data are in the same sampling rate ! Thom Le 06/03/2014 16:18, Helene Lyon-Caen a écrit : > Thomas, > we have been trying to understand with the computer engineer here. The > only differences with what is on your web site we could find is that we > use a more recent version of anaconda (1.9) and of python (2.7.6) > could that make a difference?

11 years, 1 month

Re: [MSNoise] problem accessing the subscriber list

by Thomas Lecocq

Dear Hélène, Le 06/03/2014 10:18, Helene Lyon-Caen a écrit : > > I am just starting with MSNoise and will certainly have quite a few > questions... I'm answering to this email with the mailing-list in CC :-) Go ahead with your questions ! :-) > > I have some trouble installing scikits-samplerate package, but hope to > solve this; The doc on http://msnoise.org/doc/ describes how to install it on Windows and Linux boxes. Are you on Mac ? > > Another question concerns instrument responses: they are not taken > into account at present ? No, not yet. This is definitively something I will add in the coming days/max 2 weeks, because I also need it now ! Cheers from Brussels, Thomas

11 years, 1 month

6 stations added to station list, but only 2 stations cross-correlated.

by Abbott, Robert E

Thomas, My previous thread about the AK.WAT data is only of academic interest to me as it was just a test data set downloaded form the IRIS DMC. I will be happy to work with you to find out why it is happening, however. The good news is that I am able to go through the process all the way through the stacking portion on my own data. The bad news is that it only uses 2 of the 6 stations. When I populate the station table, all six stations are located and are verified to be in the table. When I scan the archive, however, only the first two stations are present in the resultant 'data_availability' table. All the mseed files were produced identically and are in BUD format. Thanks, Rob +----------------------------+ Robert E. Abbott, Ph.D. Sandia National Laboratories Geophysics Department MS 0750 P.O. Box 5800 Albuquerque, NM 87185-0750 (505) 845-0266 +----------------------------+

11 years, 2 months

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

MSNoise