Re: [MSNoise] MSNoise Digest, Vol 47, Issue 2

Send MSNoise mailing list submissions to msnoise(a)mailman-as.oma.be To subscribe or unsubscribe via the World Wide Web, visit http://mailman-as.oma.be/mailman/listinfo/msnoise or, via email, send a message with subject or body 'help' to msnoise-request(a)mailman-as.oma.be You can reach the person managing the list at msnoise-owner(a)mailman-as.oma.be When replying, please edit your Subject line so it is more specific than "Re: Contents of MSNoise digest..." Today's Topics: 1. Stack_error (Yawar Hussain) 2. Re: Stack_error (Thomas Lecocq) 3. Mean dvv output (Colton Lynner) ---------------------------------------------------------------------- Message: 1 Date: Wed, 4 Apr 2018 10:37:23 -0300 From: Yawar Hussain &lt;yawar.pgn(a)gmail.com&gt; To: msnoise(a)mailman-as.oma.be Subject: [MSNoise] Stack_error Message-ID: <CAP7arHTi9mZ6s8KzUVzJkNB-mSSV0AYPj7FSg6DsfVMWBaAySg@ mail.gmail.com> Content-Type: text/plain; charset="UTF-8" Hello, I run stack command and received following errors. Please, have a look. I tried to fix it by editing 'my.cnf' file as below The MySQL database server configuration file. # # You can copy this to one of: # - "/etc/mysql/my.cnf" to set global options, # - "~/.my.cnf" to set user-specific options. # # One can use all long options that the program supports. # Run program with --help to get a list of available options and with # --print-defaults to see which it would actually understand and use. # # For explanations see # http://dev.mysql.com/doc/mysql/en/server-system-variables.html # # * IMPORTANT: Additional settings that can override those from this file! # The files must end with '.cnf', otherwise they'll be ignored. # !includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mysql.conf.d/ [global] sql_mode = TRADITIONAL Stack error! yawar@yawar-Vostro-5470:~/anaconda3/bin$ msnoise stack -r -m -i 10 Lets STACK ! 2018-04-03 10:59:04 [DEBUG] Starting the ref stack [] 2018-04-03 10:59:04 [DEBUG] Processing DF01_BHE:DF01_BHE-ZZ-1 Traceback (most recent call last): File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 1139, in _execute_context context) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/default.py", line 450, in do_execute cursor.execute(statement, parameters) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/cursors.py", line 166, in execute result = self._query(query) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/cursors.py", line 322, in _query conn.query(q) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 835, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 1019, in _read_query_result result.read() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 1302, in read first_packet = self.connection._read_packet() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 981, in _read_packet packet.check_error() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 393, in check_error err.raise_mysql_exception(self._data) File "/home/yawar/anaconda3/lib/python3.5/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) pymysql.err.InternalError: (1055, "Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'msnoise.jobs.ref' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/yawar/anaconda3/bin/msnoise", line 11, in <module> sys.exit(run()) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ msnoise/scripts/msnoise.py", line 614, in run cli(obj={}) File "/home/yawar/anaconda3/lib/python3.5/site-packages/click/core.py", line 716, in __call__ return self.main(*args, **kwargs) File "/home/yawar/anaconda3/lib/python3.5/site-packages/click/core.py", line 696, in main rv = self.invoke(ctx) File "/home/yawar/anaconda3/lib/python3.5/site-packages/click/core.py", line 1060, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/yawar/anaconda3/lib/python3.5/site-packages/click/core.py", line 889, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/yawar/anaconda3/lib/python3.5/site-packages/click/core.py", line 534, in invoke return callback(*args, **kwargs) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ msnoise/scripts/msnoise.py", line 301, in stack main('ref', interval) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ msnoise/s04stack.py", line 203, in main updated_days = updated_days_for_dates(db, start, end, pair.replace('_', '.'), jobtype='CC', interval=datetime.timedelta(days=interval),returndays= True) File "/home/yawar/anaconda3/lib/python3.5/site-packages/msnoise/api.py", line 1189, in updated_days_for_dates days = session.query(Job).filter(Job.pair == pair).filter(Job.day >= date1).filter(Job.day <= date2).filter(Job.jobtype == jobtype).filter(Job.lastmod >= lastmod).group_by(Job.day). order_by(Job.day).all() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/orm/query.py", line 2613, in all return list(self) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/orm/query.py", line 2761, in __iter__ return self._execute_and_instances(context) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/orm/query.py", line 2776, in _execute_and_instances result = conn.execute(querycontext.statement, self._params) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 914, in execute return meth(self, multiparams, params) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/sql/elements.py", line 323, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement compiled_sql, distilled_params File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 1146, in _execute_context context) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception exc_info File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/util/compat.py", line 202, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/util/compat.py", line 185, in reraise raise value.with_traceback(tb) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/base.py", line 1139, in _execute_context context) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ sqlalchemy/engine/default.py", line 450, in do_execute cursor.execute(statement, parameters) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/cursors.py", line 166, in execute result = self._query(query) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/cursors.py", line 322, in _query conn.query(q) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 835, in query self._affected_rows = self._read_query_result(unbuffered=unbuffered) File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 1019, in _read_query_result result.read() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 1302, in read first_packet = self.connection._read_packet() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 981, in _read_packet packet.check_error() File "/home/yawar/anaconda3/lib/python3.5/site-packages/ pymysql/connections.py", line 393, in check_error err.raise_mysql_exception(self._data) File "/home/yawar/anaconda3/lib/python3.5/site-packages/pymysql/err.py", line 107, in raise_mysql_exception raise errorclass(errno, errval) sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1055, "Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'msnoise.jobs.ref' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by") [SQL: 'SELECT jobs.ref AS jobs_ref, jobs.day AS jobs_day, jobs.pair AS jobs_pair, jobs.jobtype AS jobs_jobtype, jobs.flag AS jobs_flag, jobs.lastmod AS jobs_lastmod \nFROM jobs \nWHERE jobs.pair = %(pair_1)s AND jobs.day >= %(day_1)s AND jobs.day <= %(day_2)s AND jobs.jobtype = %(jobtype_1)s AND jobs.lastmod >= %(lastmod_1)s GROUP BY jobs.day ORDER BY jobs.day'] [parameters: {'lastmod_1': datetime.datetime(2018, 3, 24, 10, 59, 4, 395159), 'jobtype_1': 'CC', 'day_2': datetime.date(2018, 1, 1), 'pair_1': 'DF01.BHE:DF01.BHE', 'day_1': datetime.date(1970, 1, 1)}] Regards, Yawar Hussain Ph.D. Student, Geotechnical Engineering University of Brasilia, Brazil. ------------------------------ Message: 2 Date: Wed, 4 Apr 2018 17:15:04 +0200 From: Thomas Lecocq &lt;Thomas.Lecocq(a)seismology.be&gt; To: msnoise(a)mailman-as.oma.be Subject: Re: [MSNoise] Stack_error Message-ID: &lt;71175284-be10-c592-dc74-0db7ab6286c8(a)seismology.be&gt; Content-Type: text/plain; charset=utf-8; format=flowed Hi, I assume your other errors are solved, that's good news. For MySQL, did you restart the mysql or mysqld service after adapting my.cnf ? Best regards, Thomas On 04/04/2018 15:37, Yawar Hussain wrote: file! sqlalchemy/engine/base.py", sqlalchemy/engine/default.py", pymysql/cursors.py", pymysql/cursors.py", pymysql/connections.py", unbuffered=unbuffered) pymysql/connections.py", > line 1019, in _read_query_result > result.read() pymysql/connections.py", > line 1302, in read > first_packet = self.connection._read_packet() pymysql/connections.py", > line 981, in _read_packet > packet.check_error() pymysql/connections.py", > line 393, in check_error > err.raise_mysql_exception(self._data) pymysql/err.py", which > is not functionally dependent on columns in GROUP BY clause; this is > incompatible with sql_mode=only_full_group_by") > > The above exception was the direct cause of the following exception: > > Traceback (most recent call last): > File "/home/yawar/anaconda3/bin/msnoise", line 11, in <module> > sys.exit(run()) msnoise/scripts/msnoise.py", core.py", core.py", core.py", core.py", core.py", > line 534, in invoke > return callback(*args, **kwargs) msnoise/scripts/msnoise.py", > line 301, in stack > main('ref', interval) msnoise/s04stack.py", pair.replace('_', days=interval),returndays= > True) msnoise/api.py", > line 1189, in updated_days_for_dates > days = session.query(Job).filter(Job.pair == pair).filter(Job.day >= > date1).filter(Job.day <= date2).filter(Job.jobtype == > jobtype).filter(Job.lastmod >= lastmod).group_by(Job.day). > order_by(Job.day).all() sqlalchemy/orm/query.py", > line 2613, in all > return list(self) sqlalchemy/orm/query.py", > line 2761, in __iter__ > return self._execute_and_instances(context) sqlalchemy/orm/query.py", > line 2776, in _execute_and_instances > result = conn.execute(querycontext.statement, self._params) sqlalchemy/engine/base.py", > line 914, in execute > return meth(self, multiparams, params) sqlalchemy/sql/elements.py", > line 323, in _execute_on_connection > return connection._execute_clauseelement(self, multiparams, params) sqlalchemy/engine/base.py", > line 1010, in _execute_clauseelement > compiled_sql, distilled_params sqlalchemy/engine/base.py", > line 1146, in _execute_context > context) sqlalchemy/engine/base.py", > line 1341, in _handle_dbapi_exception > exc_info sqlalchemy/util/compat.py", > line 202, in raise_from_cause > reraise(type(exception), exception, tb=exc_tb, cause=cause) sqlalchemy/util/compat.py", > line 185, in reraise > raise value.with_traceback(tb) sqlalchemy/engine/base.py", sqlalchemy/engine/default.py", pymysql/cursors.py", pymysql/cursors.py", pymysql/connections.py", unbuffered=unbuffered) pymysql/connections.py", > line 1019, in _read_query_result > result.read() pymysql/connections.py", > line 1302, in read > first_packet = self.connection._read_packet() pymysql/connections.py", > line 981, in _read_packet > packet.check_error() pymysql/connections.py", > line 393, in check_error > err.raise_mysql_exception(self._data) pymysql/err.py", dependent jobs.day %(day_2)s BY 'day_1': -- Dr. Thomas Lecocq Geologist - Seismologist Seismology - Gravimetry Royal Observatory of Belgium * * * * * * * * * * --------- http://www.seismology.be http://msnoise.org http://twitter.com/#!/Seismologie_be https://www.facebook.com/seismologie.be ------------------------------ Message: 3 Date: Wed, 4 Apr 2018 14:47:22 -0700 From: Colton Lynner &lt;colton.lynner(a)gmail.com&gt; To: msnoise(a)mailman-as.oma.be Subject: [MSNoise] Mean dvv output Message-ID: &lt;65BA42B1-BB75-484F-AC31-1F5B244C2DE7(a)gmail.com&gt; Content-Type: text/plain; charset=us-ascii Hello, I have a couple questions about MSNoise. 1) Is there a way to output the mean values calculated from the DVV plots? I have found that if you use the -p option with the ALL pair, it produces a csv file of the ALL line values. The ALL and mean lines are slightly different for some frequencies, and I would like to be able to output both. Also, the documentation mentions the ALL pair, but can you offer more explanation on how that is calculated? Is it an average (stack?) of all traces? I am trying to work out why the ALL and the mean may be different. 2) Is there a plot tool to make the delay time variation plots that are shown in the documentation and in the SRL paper (Figure 4)? Thanks, Colton ------------------------------ _______________________________________________ MSNoise mailing list MSNoise(a)mailman-as.oma.be http://mailman-as.oma.be/mailman/listinfo/msnoise End of MSNoise Digest, Vol 47, Issue 2 **************************************