Re: [MSNoise] Station list

21 Sep 2016

Thank you! This addition to the my.cnf file has finally fixed my problem too!

I was almost about to reinstall Ubuntu 14, but now it seems that MySQL is working like
normal with this fix. Thanks!

~~~~~~~~~~~~~~~~~~~~~~~~~

Emily Crowder MSc BSc
PhD Student
School of Geosciences
University of Aberdeen
Aberdeen AB24 3UE
Scotland
Em: emily.crowder(a)abdn.ac.uk

~~~~~~~~~~~~~~~~~~~~~~~~~

________________________________
From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of VICIC BLAZ [PHD0900016] &lt;BLAZ.VICIC(a)phd.units.it&gt;
Sent: 20 September 2016 19:50:11
To: msnoise(a)mailman-as.oma.be
Subject: Re: [MSNoise] Station list

You were right... i found the solution:

If someone else will have this problem.

On Ubuntu 16.04 u need to add this at the end of my.cnf file:

[mysqld]
sql_mode="TRADITIONAL,NO_AUTO_CREATE_USER"

________________________________
From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of Thomas Lecocq &lt;Thomas.Lecocq(a)seismology.be&gt;
Sent: Tuesday, September 20, 2016 8:08:11 PM
To: msnoise(a)mailman-as.oma.be
Subject: Re: [MSNoise] Station list

Blaz,

The solution I gave worked for me. Ask you sysadmin or someone who knows
about linux systems, it might be a different setting on different
flavours of linux... It's not a MSNoise problem, I'll try to come up
with a workaround later but for now I'm sure you'll find a solution.

Thomas

On 20/09/2016 17:29, VICIC BLAZ [PHD0900016] wrote:
...
  The problem was actually in PyTables... I reinstalled
it with apt-get and it did not work...then i checked with anaconda. it was not installed
there, so i installed it, and now it works.

 Next problem is, as i saw the same, as Emily had last month... was it resolved?

 i try to stack and i get:

 Lets STACK !
 2016-09-20 18:21:23 [DEBUG] Starting the ref stack
 []
 2016-09-20 18:21:24 [DEBUG] Processing SL_CEY:SL_JAVS-ZZ-2
 Traceback (most recent call last):
    File "/home/blaz/miniconda2/bin/msnoise", line 11, in <module>
      sys.exit(run())
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/scripts/msnoise.py",
line 614, in run
      cli(obj={})
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line
716, in __call__
      return self.main(*args, **kwargs)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line
696, in main
      rv = self.invoke(ctx)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line
1060, in invoke
      return _process_result(sub_ctx.command.invoke(sub_ctx))
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line
889, in invoke
      return ctx.invoke(self.callback, **ctx.params)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/click/core.py", line
534, in invoke
      return callback(*args, **kwargs)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/scripts/msnoise.py",
line 301, in stack
      main('ref', interval)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/s04stack.py", line
203, in main
      updated_days = updated_days_for_dates(db, start, end, pair.replace('_',
'.'), jobtype='CC',
interval=datetime.timedelta(days=interval),returndays=True)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/api.py",
line 1189, in updated_days_for_dates
      days = session.query(Job).filter(Job.pair == pair).filter(Job.day >=
date1).filter(Job.day <= date2).filter(Job.jobtype == jobtype).filter(Job.lastmod >=
lastmod).group_by(Job.day).order_by(Job.day).all()
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2613, in all
      return list(self)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2761, in __iter__
      return self._execute_and_instances(context)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/orm/query.py",
line 2776, in _execute_and_instances
      result = conn.execute(querycontext.statement, self._params)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 914, in execute
      return meth(self, multiparams, params)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/sql/elements.py",
line 323, in _execute_on_connection
      return connection._execute_clauseelement(self, multiparams, params)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1010, in _execute_clauseelement
      compiled_sql, distilled_params
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1146, in _execute_context
      context)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1341, in _handle_dbapi_exception
      exc_info
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/util/compat.py",
line 202, in raise_from_cause
      reraise(type(exception), exception, tb=exc_tb, cause=cause)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/base.py",
line 1139, in _execute_context
      context)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/sqlalchemy/engine/default.py",
line 450, in do_execute
      cursor.execute(statement, parameters)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/cursors.py",
line 166, in execute
      result = self._query(query)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/cursors.py",
line 322, in _query
      conn.query(q)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line
835, in query
      self._affected_rows = self._read_query_result(unbuffered=unbuffered)
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line
1019, in _read_query_result
      result.read()
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line
1302, in read
      first_packet = self.connection._read_packet()
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line
981, in _read_packet
      packet.check_error()
    File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/connections.py", line
393, in check_error
      err.raise_mysql_exception(self._data)
    File "/home/blaz/miniconda2/lib/python2.7/site-packages/pymysql/err.py",
line 107, in raise_mysql_exception
      raise errorclass(errno, errval)
 sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1055, u"Expression #1 of
SELECT list is not in GROUP BY clause and contains nonaggregated column
'msnoise.jobs.ref' which is not functionally dependent on columns in GROUP BY
clause; this is incompatible with sql_mode=only_full_group_by") [SQL: u'SELECT
jobs.ref AS jobs_ref, jobs.day AS jobs_day, jobs.pair AS jobs_pair, jobs.jobtype AS
jobs_jobtype, jobs.flag AS jobs_flag, jobs.lastmod AS jobs_lastmod \nFROM jobs \nWHERE
jobs.pair = %(pair_1)s AND jobs.day >= %(day_1)s AND jobs.day <= %(day_2)s AND
jobs.jobtype = %(jobtype_1)s AND jobs.lastmod >= %(lastmod_1)s GROUP BY jobs.day ORDER
BY jobs.day'] [parameters: {u'day_2': datetime.date(2016, 9, 20),
u'lastmod_1': datetime.datetime(2016, 9, 19, 18, 21, 24, 191369),
u'day_1': datetime.date(1970, 1, 1), u'jobtype_1': 'CC',
u'pair_1': 'SL.CEY:SL.JAVS'}]

 So, as you mentioned,  i added sql_mode=TRADITIONAL line in my.cnf, and the same as for
Emily,
 sudo service mysql  restart did not work. i added  [mysql] in front of the line that i
added, and now restart works, but I get the same error as before when calling

 msnoise stack -r -m

 any idea? or should i install older version of mysql?

 thanks

 ________________________________
 From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of Thomas Lecocq &lt;Thomas.Lecocq(a)seismology.be&gt;
 Sent: Tuesday, September 20, 2016 5:10:46 PM
 To: msnoise(a)mailman-as.oma.be
 Subject: Re: [MSNoise] Station list

 Hi,

 you probably checked "keep all" parameter. it's really not properly
 tested. just avoid it for the moment.

 Thomas

 On 20/09/2016 15:50, VICIC BLAZ [PHD0900016] wrote:
  Sorry, but there is another problem...

 i ran:

 msnoise -t 2 scan_archive --init

 msnoise new_jobs

 without problems.

 then i tried with

 msnoise -t 4 compute_cc

 it starts to work, but after some time it produces this error:

 ...

 2016-09-20 16:42:18 [DEBUG] Azimuth=258.9
 Process Process-1:
 Traceback (most recent call last):
     File "/home/blaz/miniconda2/lib/python2.7/multiprocessing/process.py", line
258, in _bootstrap
       self.run()
     File "/home/blaz/miniconda2/lib/python2.7/multiprocessing/process.py", line
114, in run
       self._target(*self._args, **self._kwargs)
     File
"/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/s03compute_cc.py",
line 683, in main
       export_allcorr(db, ccfid, allcorr[ccfid])
     File "/home/blaz/miniconda2/lib/python2.7/site-packages/msnoise/api.py",
line 825, in export_allcorr
       df.to_hdf(os.path.join(path, date+'.h5'), 'data')
     File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/core/generic.py", line
1101, in to_hdf
       return pytables.to_hdf(path_or_buf, key, self, **kwargs)
     File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/io/pytables.py", line
259, in to_hdf
       complib=complib) as store:
     File
"/home/blaz/miniconda2/lib/python2.7/site-packages/pandas/io/pytables.py", line
389, in __init__
       'importing'.format(ex=str(ex)))
 ImportError: HDFStore requires PyTables, "No module named tables" problem
importing

 I tried reinstalling PyTables without luck.

 Thanks!

 ________________________________
 From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of VICIC BLAZ [PHD0900016] &lt;BLAZ.VICIC(a)phd.units.it&gt;
 Sent: Tuesday, September 20, 2016 3:43:22 PM
 To: msnoise(a)mailman-as.oma.be
 Subject: Re: [MSNoise] Station list

 Thanks, it works now.

 Blaž

 ________________________________
 From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of Thomas Lecocq &lt;Thomas.Lecocq(a)seismology.be&gt;
 Sent: Tuesday, September 20, 2016 3:01:07 PM
 To: msnoise(a)mailman-as.oma.be
 Subject: Re: [MSNoise] Station list

 Hi,

 You have to import data_structure too :

 from msnoise.data_structures import data_structure

 otherwise it'll fail (your custom.py file is not valid, fails at import).

 Thomas

 On 20/09/2016 12:59, VICIC BLAZ [PHD0900016] wrote:
  Thank you.

 Which brings me to the 2nd question...

 I tried to populate with this command, but of course it is not working...

 What I tried:

 in the folder

 msnoise/data I have put a miniseed file of one station of one day.

 Name of the miniseed is in the format:

 STA.NET.CHAN.YEAR.DAY

 as data_folder in settings I have set this folder.

 in the data_structures.py I added this line:

 data_structure['SEI'] = "STA.NET.CHAN.YEAR.DAY"

 i created a custom.py script:

 from obspy import read

 data_structure['SEI'] = "STA.NET.CHAN.YEAR.DAY"

 def populate():
        st = read("*")
        st.merge()

        stations = []

        for tr in st:
            sta = str(tr.stats.station)
            net = str(tr.stats.network)
            stations.append(str("%s_%s") % (net, sta))

        return stations

 So,now i have main folder with sub folder data, where I have miniseed file. in the main
folder I put custom.py and db.ini and, if I understand right, run msnoise populate from
here. In the browser I set data_folder to where my miniseed is and data_structure as SEI.

 When i run the populate command, i get error:

 ()
 > Populating the Station table  ()
 Can't parse the archive for format SEI !
 trying to import local parser (should return a station list)
 ()
 No file named custom.py in the /home/blaz/Documents/msnoise folder

 (custom.py IS in /home/blaz/Documents/msnoise)
 I hope I was clear enough.

 Thanks!

 ________________________________
 From: msnoise-bounces(a)mailman-as.oma.be &lt;msnoise-bounces(a)mailman-as.oma.be&gt; on
behalf of Thomas Lecocq &lt;Thomas.Lecocq(a)seismology.be&gt;
 Sent: Tuesday, September 20, 2016 12:56:56 PM
 To: msnoise(a)mailman-as.oma.be
 Subject: Re: [MSNoise] Station list

 Hi,

 Yes, this bug was recently reported by another user and I haven't had
 time to fix it.

 The best way to add the station currently is to use the "msnoise
 populate" command.

 Cheers,

 Thomas

 On 20/09/2016 11:51, VICIC BLAZ [PHD0900016] wrote:
  Dear all

 I would like to try to use msnoise. I installed it on Ubuntu 16.04, and as far as I know,
everything was installed as planed.

 Now, I open msnoise admin and try to manually add stations in the browser. I set NET,
STA, X, Y, ALT, COORD and click save. When I do that, I get a lot of errors:

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
2000, in __call__

 return self.wsgi_app(environ, start_response)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1991, in wsgi_app

 response = self.make_response(self.handle_exception(e))

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1567, in handle_exception

 reraise(exc_type, exc_value, tb)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1988, in wsgi_app

 response = self.full_dispatch_request()

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1641, in full_dispatch_request

 rv = self.handle_user_exception(e)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1544, in handle_user_exception

 reraise(exc_type, exc_value, tb)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1639, in full_dispatch_request

 rv = self.dispatch_request()

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask/app.py", line
1625, in dispatch_request

 return self.view_functions[rule.endpoint](**req.view_args)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/base.py",
line 69, in inner

 return self._run_view(f, *args, **kwargs)

       *
 File "/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/base.py",
line 368, in _run_view

 return fn(self, *args, **kwargs)

       *
 File
"/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/model/base.py",
line 1920, in create_view

 model = self.create_model(form)

       *
 File
"/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py",
line 1028, in create_model

 if not self.handle_view_exception(ex):

       *
 File
"/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py",
line 1011, in handle_view_exception

 return super(ModelView, self).handle_view_exception(exc)

       *
 File
"/home/blaz/miniconda2/lib/python2.7/site-packages/flask_admin/contrib/sqla/view.py",
line 1022, in create_model

 model = self.model()

 TypeError: __init__() takes exactly 9 arguments (1 given)

 What am I doing wrong?

 Thanks,

 Blaz
 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise 
_______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise
 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise 
_______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise
 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise
 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise 
_______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise
 _______________________________________________
 MSNoise mailing list
 MSNoise(a)mailman-as.oma.be
 http://mailman-as.oma.be/mailman/listinfo/msnoise 
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise
_______________________________________________
MSNoise mailing list
MSNoise(a)mailman-as.oma.be
http://mailman-as.oma.be/mailman/listinfo/msnoise

The University of Aberdeen is a charity registered in Scotland, No SC013683.
Tha Oilthigh Obar Dheathain na charthannas clàraichte ann an Alba, Àir. SC013683.

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Re: [MSNoise] Station list