New subject: MSNoise Digest, Vol 5, Issue 6

21 Mar 2014

Hi Thomas, 

I switch from sqlite to mysql and now the s02new_jobs.py works well. But I got this really
weird error when I run:

Estebans-MacBook-Pro:MSN ech$ python s05compute_mwcs.py 
/Users/ech/anaconda/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11:
ModuleDeprecationWarning: The oldnumeric module will be dropped in Numpy 1.9
  warnings.warn(_msg, ModuleDeprecationWarning)
2014-03-20 19:57:52,503 [INFO] *** Starting: Compute MWCS ***
2014-03-20 19:57:52,599 [INFO] We will recompute all MWCS based on the new REF for
YZ.HRIU:YZ.INDI
2014-03-20 19:57:52,615 [INFO] We will recompute all MWCS based on the new REF for
YZ.HRIU:YZ.NARJ
2014-03-20 19:57:52,631 [INFO] We will recompute all MWCS based on the new REF for
YZ.INDI:YZ.NARJ
2014-03-20 19:57:52,714 [INFO] There are MWCS jobs for some days to recompute for
YZ.HRIU:YZ.INDI
2014-03-20 19:57:52,721 [DEBUG] Processing MWCS for: YZ_HRIU_YZ_INDI.ZZ.01 - 2012-01-01 -
05 days
Traceback (most recent call last):
  File "s05compute_mwcs.py", line 118, in <module>
    cur, ref, f.mwcs_low, f.mwcs_high, goal_sampling_rate, -maxlag, f.mwcs_wlen,
f.mwcs_step)
  File "/Users/ech/sismo/MSN/MWCS.py", line 144, in mwcs
    res = sm.regression.linear_model.WLS(phi, v, w**2).fit()
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py",
line 381, in __init__
    weights=weights, hasconst=hasconst)
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py",
line 79, in __init__
    super(RegressionModel, self).__init__(endog, exog, **kwargs)
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py",
line 136, in __init__
    super(LikelihoodModel, self).__init__(endog, exog, **kwargs)
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py",
line 52, in __init__
    self.data = handle_data(endog, exog, missing, hasconst, **kwargs)
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line
397, in handle_data
    return klass(endog, exog=exog, missing=missing, hasconst=hasconst, **kwargs)
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line
78, in __init__
    self._check_integrity()
  File
"/Users/ech/anaconda/lib/python2.7/site-packages/statsmodels/base/data.py", line
246, in _check_integrity
    if len(self.exog) != len(self.endog):
TypeError: len() of unsized object

Thanks in advance, 

Esteban

On Mar 14, 2014, at 5:00 AM, msnoise-request(a)mailman-as.oma.be wrote:

...
  Send MSNoise mailing list submissions to
 	msnoise(a)mailman-as.oma.be

 To subscribe or unsubscribe via the World Wide Web, visit
 	http://mailman-as.oma.be/mailman/listinfo/msnoise
 or, via email, send a message with subject or body 'help' to
 	msnoise-request(a)mailman-as.oma.be

 You can reach the person managing the list at
 	msnoise-owner(a)mailman-as.oma.be

 When replying, please edit your Subject line so it is more specific
 than "Re: Contents of MSNoise digest..."

 Today's Topics:

   1. Re: Found 0 new Jobs to do (Thomas Lecocq)

 ----------------------------------------------------------------------

 Message: 1
 Date: Thu, 13 Mar 2014 13:37:51 +0100
 From: Thomas Lecocq &lt;thomas.lecocq(a)oma.be&gt;
 To: msnoise(a)mailman-as.oma.be
 Subject: Re: [MSNoise] Found 0 new Jobs to do
 Message-ID: &lt;5321A69F.5090303(a)oma.be&gt;
 Content-Type: text/plain; charset=windows-1252; format=flowed

 Hi Esteban,

 If you use sqlite, you cannot use more than 1 thread, otherwise there 
 are big chances some data will be lost. So option "-t 1" is compulsory.

 With the new scan_archive code, there is no longer visual ouptut of the 
 scans, that is a problem I'm trying to solve soon.

 When you run the scan_archive -i -t1, it looks like your archive takes 
 some time to be scanned (72s), so I assume the scan is OK. Can you 
 confirm that after this step, the data_availability table contains 
 metadata for the station/files ?

 Then, what could happen is that you haven't properly configured the 
 start and end-times in the configurator, so no new jobs are found.

 Thomas

 Le 12/03/2014 20:35, Esteban Chaves a ?crit :
  Hi Tomas et al;

 I got the next problem:

 Estebans-MacBook-Pro:MSNoise-master ech$ python s002populate_station_table.py

 > Populating the Station table  Adding:
YZ INDI
 Adding: YZ JUDI
 Adding: YZ SAJU
 Estebans-MacBook-Pro:MSNoise-master ech$ python s01scan_archive.py -i -t 2
 2014-03-12 12:14:23,805 [INFO] *** Starting: Scan Archive ***
 2014-03-12 12:14:23,816 [INFO] Initializing (should be run only once)
 2014-03-12 12:14:23,816 [INFO] Will work on 2 threads
 2014-03-12 12:14:23,830 [INFO] Started: /Users/ech/gap/YZ/INDI
 2014-03-12 12:14:23,833 [INFO] Started: /Users/ech/gap/YZ/INDI
 2014-03-12 12:14:23,834 [INFO] child process calling self.run()
 2014-03-12 12:14:23,836 [INFO] child process calling self.run()
 2014-03-12 12:14:41,157 [INFO] process shutting down
 2014-03-12 12:14:41,157 [INFO] process exiting with exitcode 0
 2014-03-12 12:14:41,159 [INFO] process shutting down
 2014-03-12 12:14:41,159 [INFO] process exiting with exitcode 0
 2014-03-12 12:14:41,196 [INFO] Started: /Users/ech/gap/YZ/INDI
 2014-03-12 12:14:41,200 [INFO] child process calling self.run()
 2014-03-12 12:14:41,200 [INFO] Started: /Users/ech/gap/YZ/JUDI
 2014-03-12 12:14:41,204 [INFO] child process calling self.run()
 2014-03-12 12:14:56,235 [INFO] process shutting down
 2014-03-12 12:14:56,236 [INFO] process exiting with exitcode 0
 2014-03-12 12:14:56,275 [INFO] Started: /Users/ech/gap/YZ/JUDI
 2014-03-12 12:14:56,279 [INFO] child process calling self.run()
 2014-03-12 12:14:57,349 [INFO] process shutting down
 2014-03-12 12:14:57,349 [INFO] process exiting with exitcode 0
 2014-03-12 12:14:57,372 [INFO] Started: /Users/ech/gap/YZ/JUDI
 2014-03-12 12:14:57,375 [INFO] child process calling self.run()
 2014-03-12 12:15:09,552 [INFO] process shutting down
 2014-03-12 12:15:09,552 [INFO] process exiting with exitcode 0
 2014-03-12 12:15:09,578 [INFO] Started: /Users/ech/gap/YZ/SAJU
 2014-03-12 12:15:09,581 [INFO] child process calling self.run()
 2014-03-12 12:15:10,519 [INFO] process shutting down
 2014-03-12 12:15:10,519 [INFO] process exiting with exitcode 0
 2014-03-12 12:15:10,539 [INFO] Started: /Users/ech/gap/YZ/SAJU
 2014-03-12 12:15:10,542 [INFO] child process calling self.run()
 2014-03-12 12:15:24,492 [INFO] process shutting down
 2014-03-12 12:15:24,492 [INFO] process exiting with exitcode 0
 2014-03-12 12:15:24,493 [INFO] process shutting down
 2014-03-12 12:15:24,494 [INFO] process exiting with exitcode 0
 2014-03-12 12:15:24,519 [INFO] Started: /Users/ech/gap/YZ/SAJU
 2014-03-12 12:15:24,522 [INFO] child process calling self.run()
 2014-03-12 12:15:36,690 [INFO] process shutting down
 2014-03-12 12:15:36,691 [INFO] process exiting with exitcode 0
 2014-03-12 12:15:36,709 [INFO] *** Finished: Scan Archive ***
 2014-03-12 12:15:36,710 [INFO] It took 72.91 seconds
 2014-03-12 12:15:36,710 [INFO] process shutting down
 Estebans-MacBook-Pro:MSNoise-master ech$ python s02new_jobs.py
 2014-03-12 12:15:53,224 [INFO] *** Starting: New Jobs ***
 2014-03-12 12:15:54,548 [DEBUG] Found 0 new jobs to do
 2014-03-12 12:15:54,664 [INFO] *** Finished: New Jobs ***

 My data is in BUD format and I?m using the last MASTER release of MSNoise. This is the
first time that I got this kind of problem. If you see my msnoise.sqlite table
 you will see that I?m trying to process data starting from day 2012-286 and basically (I
think) it cannot initialize the jobs after scan. The weird thing is that using the
previous days, say from May 2012 till September 2012 the programs works very well.

 I appreciate your help;

 Esteban
 PhD Student in Seismology
 University of California, Santa Cruz

 On Mar 9, 2014, at 5:00 AM, msnoise-request(a)mailman-as.oma.be wrote:

  Send MSNoise mailing list submissions to
 	msnoise(a)mailman-as.oma.be

 To subscribe or unsubscribe via the World Wide Web, visit
 	http://mailman-as.oma.be/mailman/listinfo/msnoise
 or, via email, send a message with subject or body 'help' to
 	msnoise-request(a)mailman-as.oma.be

 You can reach the person managing the list at
 	msnoise-owner(a)mailman-as.oma.be

 When replying, please edit your Subject line so it is more specific
 than "Re: Contents of MSNoise digest..."

 Today's Topics:

   1. Fwd: MSNoise: filters (Thomas Lecocq)

 ----------------------------------------------------------------------

 Message: 1
 Date: Sat, 08 Mar 2014 16:56:15 +0100
 From: Thomas Lecocq &lt;thomas.lecocq(a)seismology.be&gt;
 To: Python Package for Monitoring Seismic Velocity Changes using
 	Ambient	Seismic Noise &lt;msnoise(a)mailman-as.oma.be&gt;
 Subject: [MSNoise] Fwd: MSNoise: filters
 Message-ID: &lt;531B3D9F.3040003(a)seismology.be&gt;
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed

 Hi H?l?ne,

 The filters need to be defined in the configurator. Filter bounds define
 the bandpass that will whitened. Around this whitened part, the spectra
 will be tapered to 0.+0j.

 Indeed, I've rechecked the doc, it's not very obvious. Thanks for
 pointing this out :) I'll try to spend some time on the doc soon.

 Briefly: each filter is defined with:
 low : lower frequency of the bandpass
 high : upper ...

 mwcs_low : slightly higher than low, it is the lower frequency for which
 MWCS' weighted linear regression will be estimated
 mwcs_high : slightly lower than high,...

 mwcs_wlen : window length for computing the relative delay in the MWCS
 mwcs_step : step between two windows. if equal to mwcs_wlen: no overlap.

 rms_threshold: not used any more, should disappear, set it to 0.0.

 Hope this helps, let me know (via this mailing list, svp.)

 Thom

 -------- Message original --------
 Sujet: 	MSNoise: filters
 Date : 	Sat, 8 Mar 2014 14:14:33 +0100

 Pour : 	Thomas Lecocq &lt;thomas.lecocq(a)oma.be&gt;

 Thomas, we solved the problem with scikits_samplerate apprently related to
 some export in my .bashrc.
 I am now  at s03compute_cc.py and get an error because I did not define any
 filters: apparently this is not implementted yet (?) , to what these filters
 correspond?
 Cheers
 Helene

 -----Message d'origine-----
 From: Thomas Lecocq
 Sent: Thursday, March 06, 2014 5:45 PM
 To: Helene.Lyon-Caen(a)ens.fr
 Subject: Re: scikits

 H?l?ne,

 You could remove the dependency to scikits by using only Decimation, if
 all your data are in the same sampling rate !

 Thom

 Le 06/03/2014 16:18, Helene Lyon-Caen a ?crit :
  Thomas,
 we have been trying to understand with the computer engineer here. The
 only differences with what is on your web site we could find is that we
 use a more recent version of anaconda (1.9) and of python (2.7.6)
 could that make a difference?  

 ------------------------------

 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise

 End of MSNoise Digest, Vol 5, Issue 4
 ************************************* 
_______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise  

 ------------------------------

 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise

 End of MSNoise Digest, Vol 5, Issue 6
 ************************************* 

Re: [MSNoise] MSNoise Digest, Vol 5, Issue 6