2 Jun
2017
2 Jun
'17
6:55 p.m.
Dear Christos and Lukas,
first of all thanks for you help! I really apreciate it!
Christos: I have a server with 40 cores, so I think the *-t 20* flag is
working fine!
Lukas: I tried to do a *pip install msnoise*, but I don't think it is
actually upgrading MSNoise, the output is:
*Requirement already satisfied: msnoise in
/usr/local/anaconda/lib/python2.7/site-packages*
*Requirement already satisfied: setuptools in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: numpy>=1.0 in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: scipy in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: pandas in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: matplotlib in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: statsmodels in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: sqlalchemy in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: obspy in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: click in
/usr/local/anaconda/lib/python2.7/site-packages/click-4.0-py2.7.egg (from
msnoise)*
*Requirement already satisfied: pymysql in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: flask in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: flask-admin in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: multiprocessing_logging in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: markdown in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: folium in
/usr/local/anaconda/lib/python2.7/site-packages (from msnoise)*
*Requirement already satisfied: six>=1.10 in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: python-dateutil in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: functools32 in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: subprocess32 in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: pytz in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: cycler>=0.10 in
/usr/local/anaconda/lib/python2.7/site-packages/cycler-0.10.0-py2.7.egg
(from matplotlib->msnoise)*
*Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 in
/usr/local/anaconda/lib/python2.7/site-packages (from matplotlib->msnoise)*
*Requirement already satisfied: future>=0.12.4 in
/usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)*
*Requirement already satisfied: lxml in
/usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)*
*Requirement already satisfied: decorator in
/usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)*
*Requirement already satisfied: requests in
/usr/local/anaconda/lib/python2.7/site-packages (from obspy->msnoise)*
*Requirement already satisfied: wtforms in
/usr/local/anaconda/lib/python2.7/site-packages (from flask-admin->msnoise)*
*Requirement already satisfied: Jinja2 in
/usr/local/anaconda/lib/python2.7/site-packages (from folium->msnoise)*
This is the second time I do *msnoise -t 20 compute_mwcs*, last time it
worked fine but I needed to change *dtt_lag* to "dynamic". So when I tried
to rerun* msnoise -t 20 compute_mwcs* with the new parameter this error
message appears. What I've done is run the process in the morning, all day
it works fine, but the next morning I always find that error message.
Thanks again!
Oscar
On Fri, Jun 2, 2017 at 8:39 AM, Lukas Preiswerk <preiswerk(a)vaw.baug.ethz.ch>
wrote:
Hi Oscar
The problem is that “MySQL server has gone away”, so it can’t update
the database after finishing the MWCS job. We fixed this in the newest
version, so you should indeed update.
I think that updating is done by “pip install msnoise”, and then for
each project “msnoise upgrade_db” (see the release notes
http://msnoise.org/doc/releasenotes/msnoise-1.5.html ). Tom can
clarify if this method I just described is actually the way to go…
Best, Lukas
2017-06-02 15:20 GMT+02:00 Christos Evangelidis <cevan(a)noa.gr>gr>:
Dear Oscar,
Can you really run 20 instances of the code (-t 20) ? I find the -t 20
rather high, but your machine might support it..
Regards,
Christos
Στις 1 Ιουν 2017 6:01 μμ, ο χρήστης "Oscar Alberto Castro Artola" <
oscar.cas.art(a)gmail.com> έγραψε:
Hello,
I have 10 years of data for 13 stations, I have computed the CCF and now
it
is turn to compute the MWCS. When I do *msnoise
-t 20 compute_mwcs *this
error appears:
*Process Process-4:*
*Traceback (most recent call last):*
* File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py",
line 258, in _bootstrap*
* self.run()*
* File "/usr/local/anaconda/lib/python2.7/multiprocessing/process.py",
line 114, in run*
* self._target(*self._args, **self._kwargs)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/msnoise/
s05compute_mwcs.py",
line 150, in main*
* update_job(db, day, pair, jobtype='DTT', flag='D')*
* File "/usr/local/anaconda/lib/python2.7/site-packages/
msnoise/api.py",
line 615, in update_job*
* job = session.query(Job).filter(Job.day == day).filter(Job.pair ==
pair).filter(Job.jobtype == jobtype).first()*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/orm/query.py",
line 2755, in first*
* ret = list(self[0:1])*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/orm/query.py",
line 2547, in __getitem__*
* return list(res)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/orm/query.py",
line 2855, in __iter__*
* return self._execute_and_instances(context)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/orm/query.py",
line 2878, in _execute_and_instances*
* result = conn.execute(querycontext.statement, self._params)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/engine/base.py",
line 945, in execute*
* return meth(self, multiparams, params)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/
sql/elements.py",
line 263, in _execute_on_connection*
* return connection._execute_clauseelement(self, multiparams,
params)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/engine/base.py",
> line 1053, in _execute_clauseelement*
> * compiled_sql, distilled_params*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/engine/base.py",
> line 1189, in _execute_context*
> * context)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/engine/base.py",
> line 1402, in _handle_dbapi_exception*
> * exc_info*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/util/compat.py",
> line 203, in raise_from_cause*
> * reraise(type(exception), exception, tb=exc_tb, cause=cause)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
sqlalchemy/engine/base.py",
line 1182, in _execute_context*
* context)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/sqlalchemy/
engine/default.py",
line 470, in do_execute*
* cursor.execute(statement, parameters)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py",
line
158, in execute*
* result = self._query(query)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/pymysql/cursors.py",
line
> 308, in _query*
> * conn.query(q)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
pymysql/connections.py",
> line 819, in query*
> * self._execute_command(COMMAND.COM_QUERY, sql)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
pymysql/connections.py",
> line 1037, in _execute_command*
> * self._write_bytes(packet)*
* File
"/usr/local/anaconda/lib/python2.7/site-packages/
pymysql/connections.py",
line 989, in _write_bytes*
* raise err.OperationalError(2006, "MySQL server has gone away (%r)" %
(e,))*
*OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server
has
gone away (error(32, 'Broken
pipe'))") [SQL: u'SELECT jobs.ref AS
jobs_ref,
jobs.day AS jobs_day, jobs.pair AS jobs_pair,
jobs.jobtype AS
jobs_jobtype,
jobs.flag AS jobs_flag, jobs.lastmod AS
jobs_lastmod \nFROM jobs \nWHERE
jobs.day = %(day_1)s AND jobs.pair = %(pair_1)s AND jobs.jobtype =
%(jobtype_1)s \n LIMIT %(param_1)s'] [parameters: {u'param_1': 1,
u'day_1':
'2009-02-16', u'jobtype_1':
'DTT', u'pair_1': 'IG.ARIG:IG.MOIG'}]*
At the beginning everything works fine, but because it takes a long time
to
make the computation (around 5 days), every time
that I come back to see
the results this happens. I am working with MSNoise 1.4 and I don't know
how to upgrade to MSNoise 1.5.
Does anyone know how to fix this?
Thanks in advance!
Oscar
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise