9 May
2014
9 May
'14
2:18 a.m.
Dear All,
The whole cron.sh was humming a long nicely on our MSNoise network, until I
changed from stacking 1,2 and 5 days (initially) to 1,2,3,4,5,30 days. Now,
all steps seem to run UNTIL the plot command. Here is the error (after
putting a print statement on the cause of the problem, df):
(I run as sudo, because root is running the cron.sh overnight:)
[kasper@localhost MSNoise-master]$ sudo python s07plot_dtt.py
loading 1 days
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Traceback (most recent call last):
File "s07plot_dtt.py", line 102, in <module>
alldf = alldf.append(df)
File "/usr/lib64/python2.7/site-packages/pandas/core/frame.py", line
4266, in append
verify_integrity=verify_integrity)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
883, in concat
return op.get_result()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
964, in get_result
new_data = self._get_concatenated_data()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1007, in _get_concatenated_data
new_data[item] = self._concat_single_item(rdata, item)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1094, in _concat_single_item
return com._concat_compat(to_concat, axis=self.axis - 1)
File "/usr/lib64/python2.7/site-packages/pandas/core/common.py", line
1191, in _concat_compat
axis=axis)
ValueError: need at least one array to concatenate
12 May
12 May
9:02 a.m.
Kasper,
When you changed the mov_stack value, did you re-run MSNoise "from
scratch". The thing is, if jobs have already been done, say, for January
1 -> January 10, then you modify the mov_stack, and run s04_stack.py a
few days after, Jan1->10 will not be stacked for the 30 day mov_stack.
you have, at least, to run s04stack.py with a "--interval X" (X > 1,
your case probably 20 or 30 max), so the CC jobs in the last 30 days
will be re-considered for stacking.
As of the plot problem, I've noticed a slight bug in the get_stations()
in the database_tools, but that should only affect the results iif the
stations were input manually in the database and thus not necessarily in
alphabetical order:
(bugfix:https://github.com/ROBelgium/MSNoise/commit/0dca3cfe101c56f3ed8f6e53…).
In your case, it looks like the 1-day data is not loaded, could you
confirm the data is present in the folders/files archive ? You could
also print the "day" variable, to know which file is "empty" and
causes
the bug. Normally, empty dtt files should not exist...
Let me know,
Tom
Le 09/05/2014 02:18, Kasper van Wijk a écrit :
Dear All,
The whole cron.sh was humming a long nicely on our MSNoise network, until I
changed from stacking 1,2 and 5 days (initially) to 1,2,3,4,5,30 days. Now,
all steps seem to run UNTIL the plot command. Here is the error (after
putting a print statement on the cause of the problem, df):
(I run as sudo, because root is running the cron.sh overnight:)
[kasper@localhost MSNoise-master]$ sudo python s07plot_dtt.py
loading 1 days
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Traceback (most recent call last):
File "s07plot_dtt.py", line 102, in <module>
alldf = alldf.append(df)
File "/usr/lib64/python2.7/site-packages/pandas/core/frame.py", line
4266, in append
verify_integrity=verify_integrity)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
883, in concat
return op.get_result()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
964, in get_result
new_data = self._get_concatenated_data()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1007, in _get_concatenated_data
new_data[item] = self._concat_single_item(rdata, item)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1094, in _concat_single_item
return com._concat_compat(to_concat, axis=self.axis - 1)
File "/usr/lib64/python2.7/site-packages/pandas/core/common.py", line
1191, in _concat_compat
axis=axis)
ValueError: need at least one array to concatenate
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
1:13 p.m.
Kasper:
One extra note,
how many pairs are configured ? If you have only 1 pair, then the ALL
line will not be calculated, resulting in "missing points" in the final
dt/t red curve.
Thomas
Le 12 May 2014 09:02:54, Thomas Lecocq a écrit :
Kasper,
When you changed the mov_stack value, did you re-run MSNoise "from
scratch". The thing is, if jobs have already been done, say, for
January 1 -> January 10, then you modify the mov_stack, and run
s04_stack.py a few days after, Jan1->10 will not be stacked for the 30
day mov_stack. you have, at least, to run s04stack.py with a
"--interval X" (X > 1, your case probably 20 or 30 max), so the CC
jobs in the last 30 days will be re-considered for stacking.
As of the plot problem, I've noticed a slight bug in the
get_stations() in the database_tools, but that should only affect the
results iif the stations were input manually in the database and thus
not necessarily in alphabetical order:
(bugfix:https://github.com/ROBelgium/MSNoise/commit/0dca3cfe101c56f3ed8f6e53…).
In your case, it looks like the 1-day data is not loaded, could you
confirm the data is present in the folders/files archive ? You could
also print the "day" variable, to know which file is "empty" and
causes the bug. Normally, empty dtt files should not exist...
Let me know,
Tom
Le 09/05/2014 02:18, Kasper van Wijk a écrit :
Dear All,
The whole cron.sh was humming a long nicely on our MSNoise network,
until I
changed from stacking 1,2 and 5 days (initially) to 1,2,3,4,5,30
days. Now,
all steps seem to run UNTIL the plot command. Here is the error (after
putting a print statement on the cause of the problem, df):
(I run as sudo, because root is running the cron.sh overnight:)
[kasper@localhost MSNoise-master]$ sudo python s07plot_dtt.py
loading 1 days
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Empty DataFrame
Columns: [A, EA, EM, EM0, M, M0, Pairs]
Index: []
Traceback (most recent call last):
File "s07plot_dtt.py", line 102, in <module>
alldf = alldf.append(df)
File "/usr/lib64/python2.7/site-packages/pandas/core/frame.py", line
4266, in append
verify_integrity=verify_integrity)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
883, in concat
return op.get_result()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
964, in get_result
new_data = self._get_concatenated_data()
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1007, in _get_concatenated_data
new_data[item] = self._concat_single_item(rdata, item)
File "/usr/lib64/python2.7/site-packages/pandas/tools/merge.py", line
1094, in _concat_single_item
return com._concat_compat(to_concat, axis=self.axis - 1)
File "/usr/lib64/python2.7/site-packages/pandas/core/common.py", line
1191, in _concat_compat
axis=axis)
ValueError: need at least one array to concatenate
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
4009
days inactive
4012
days old
2 comments
2 participants
participants (2)
-
Kasper van Wijk -
Thomas Lecocq