Hi Oscar,
(cc to the mailing list, it can be very useful to others too)
Le 13/05/2014 17:11, Oscar Alberto Castro Artola a écrit :
Hello Thomas,
I have checked for CCF for 2006-01-01 and is not empty. Everything
seems to be alright! Although I realized that when my filters are too
narrow I get this error:
*oscar@bayta:~/sweet_noise$ py s05compute_mwcs.py*
2014-05-13 10:03:03,361 [INFO] We will recompute all MWCS based on the
new REF for TO.BUCU:TO.XALI
2014-05-13 10:03:03,798 [INFO] We will recompute all MWCS based on the
new REF for TO.PALM:TO.PLLI
2014-05-13 10:03:04,213 [INFO] We will recompute all MWCS based on the
new REF for TO.PALM:TO.XALI
2014-05-13 10:03:04,609 [INFO] We will recompute all MWCS based on the
new REF for TO.PLLI:TO.XALI
2014-05-13 10:03:05,989 [INFO] There are MWCS jobs for some days to
recompute for TO.BUCU:TO.XALI
2014-05-13 10:03:06,009 [DEBUG] Processing MWCS for:
TO_BUCU_TO_XALI.ZZ.04 - 2006-01-06 - 01 days
/home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py:55:
RuntimeWarning: Mean of empty slice.
warnings.warn("Mean of empty slice.", RuntimeWarning)
/home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py:77:
RuntimeWarning: Degrees of freedom <= 0 for slice
warnings.warn("Degrees of freedom <= 0 for slice", RuntimeWarning)
Traceback (most recent call last):
File "s05compute_mwcs.py", line 122, in <module>
cur, ref, f.mwcs_low, f.mwcs_high, goal_sampling_rate, -maxlag,
f.mwcs_wlen, f.mwcs_step)
File "/home/oscar/sweet_noise/MWCS.py", line 145, in mwcs
res = sm.regression.linear_model.WLS(phi, v, w**2).fit()
File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py",
line 381, in __init__
weights=weights, hasconst=hasconst)
File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py",
line 79, in __init__
super(RegressionModel, self).__init__(endog, exog, **kwargs)
File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/base/model.py",
line 137, in __init__
self.initialize()
File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/regression/linear_model.py",
line 88, in initialize
self.rank = rank(self.exog)
File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/tools/tools.py",
line 381, in rank
D = svdvals(X)
File
"/home/oscar/anaconda/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
line 146, in svdvals
check_finite=check_finite)
File
"/home/oscar/anaconda/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py",
line 100, in svd
full_matrices=full_matrices, overwrite_a=overwrite_a)
ValueError: failed to create intent(cache|hide)|optional array-- must
have defined dimensions but got (0,)
That filter is between 0.04 - 0.046 Hz.
1/ the reason for the first error (filters too narrow) is that you don't
have enough data points for doing weighted linear regression. If you use
a 10 seconds MWCS_wlen at 20Hz, you have 200 points of data to compute.
The FFT is done on the next power of 2 samples, so 256 here. Meaning,
you have 256 FFT datapoints between 0 and Nyquist=10Hz, or a frequency
step of 0.039 Hz. You filter is 0.006 Hz wide, so, at best, you'll have
1 data point (+ [0,0]) to compute the phase shift, this is not enough.
If you want to use such a narrow band, you should either CC at 100 Hz,
or edit the s05 file to padd the input data with zeros, in order to have
more resolution on the FFT (go up to the second or third next power of 2).
When my filters are broader there is no problem with
/s05compute_mwcs.py/, but when I try to run /s06compute_dtt.py/ I get
this:
*oscar@bayta:~/sweet_noise$ py s06compute_dtt.py *
*/home/oscar/anaconda/lib/python2.7/site-packages/numpy/oldnumeric/__init__.py:11:
ModuleDeprecationWarning: The oldnumeric module will be dropped in
Numpy 1.9*
* warnings.warn(_msg, ModuleDeprecationWarning)*
*2014-05-13 10:07:17,935 [INFO] *** Starting: Compute DT/T ****
*/home/oscar/anaconda/lib/python2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py:991:
UserWarning: /home/oscar/.python-eggs is writable by group/others and
vulnerable to attack when used with get_resource_filename. Consider a
more secure location (set with .set_extraction_path or the
PYTHON_EGG_CACHE environment variable).*
*2014-05-13 10:07:18,139 [DEBUG] Found 731 updated days*
*2014-05-13 10:07:18,203 [INFO] Loading mov=1 days for filter=12*
*2014-05-13 10:07:18,204 [DEBUG] Processing 2005-01-01*
*2014-05-13 10:07:18,216 [DEBUG] Processing 2005-01-02*
*2014-05-13 10:07:18,222 [DEBUG] Processing 2005-01-03*
*2014-05-13 10:07:18,228 [DEBUG] Processing 2005-01-04*
*2014-05-13 10:07:18,235 [DEBUG] Processing 2005-01-05*
*2014-05-13 10:07:18,243 [DEBUG] Processing 2005-01-06*
*2014-05-13 10:07:18,250 [DEBUG] Processing 2005-01-07*
*2014-05-13 10:07:18,257 [DEBUG] Processing 2005-01-08*
*2014-05-13 10:07:18,263 [DEBUG] Processing 2005-01-09*
*2014-05-13 10:07:18,271 [DEBUG] Processing 2005-01-10*
*2014-05-13 10:07:18,278 [DEBUG] Processing 2005-01-11*
*2014-05-13 10:07:18,284 [DEBUG] Processing 2005-01-12*
*Traceback (most recent call last):*
* File "s06compute_dtt.py", line 304, in <module>*
* VecXfilt, prepend=False)*
* File
"/home/oscar/anaconda/lib/python2.7/site-packages/statsmodels/tools/tools.py",
line 289, in add_constant*
* var0 = data.var(0) == 0*
* File
"/home/oscar/anaconda/lib/python2.7/site-packages/numpy/core/_methods.py",
line 111, in _var*
* ret = ret.dtype.type(ret / rcount)*
*AttributeError: 'float' object has no attribute 'dtype'*
2/ the second error is a bug in numpy 1.8.0, you should try to update
numpy to 1.8.1 ! This is not MSNoise :-)
Best regards,
Thomas
Thomas Lecocq