Hi Thomas,
As far as I understand you haven't changed anything since then in the code
concerning the pre-processing when computing autocorrelations. Right?
Greetings from Athens,
Christos
From: Thomas Lecocq <thomas.lecocq@...
<http://gmane.org/get-address.php?address=thomas.lecocq%2d7ewb5K7nbqnl%2fIpH…>
>
Subject: Re: Autocorrelations in MSNoise
<http://news.gmane.org/find-root.php?message_id=54DB931F.5080705%40seismolog…>
Newsgroups: gmane.science.geophysics.msnoise
<http://news.gmane.org/gmane.science.geophysics.msnoise>
Date: 2015-02-11 17:36:31 GMT (1 year, 14 weeks, 6 days, 11 hours and 16
minutes ago)
Hello Aurélien,
Le 11/02/2015 18:30, Aurélien Mordret a écrit :
> Hi Thomas,
>
> What is the pre-processing of the noise when computing autocorrelations? Is
> it the same as for the cross-correlations?
Yeah, currently it's the same, WHICH IS NOT GOOD ! (that's why it's
written somewhere NOT TO USE AUTOCORR), but that might not be trivial to
find...
Your suggestion makes absolutely sense ! Are you used to github etc ? If
yes, please fork and PR an updated version of whiten ! (including a
autocorr=False flag in the def, maybe?)!
Regards from Brussels
Thomas
>
> If yes, would it be better to not fully whiten the data before
> autocorrelation as the only information comes from the amplitude spectrum
> (and not the phase)?
>
> I suggest to modify a bit the whiten.py function to keep the structure of
> the whole code similar. Following Gorbatov et al. (2013), instead of
> putting the amplitude spectrum =1 (with tapper) I would use a normalisation
> with a water-level in the form
>
> A(freq) = A(freq)/max(A(freq),c*max(A(freq)))
>
> With A the amplitude spectrum and c the water-level factor (which is
> typically on the order of 1%).
>
> It should not change the results for the cross-correlations and improve the
> autocorrelations.
>
> Does it make sense or did I missed something?
>
> Cheers,
>
> Aurelien
>
> REF:
>
> Gorbatov, A., Saygin, E., & Kennett, B. L. N. (2013). Crustal properties
> from seismic station autocorrelograms. *Geophysical Journal International*,
> **192**(2), 861-870.
>
Dear Thomas and msnoise users
I am trying for the very first time to install and run msnoise on my linux
machine (ubuntu 12.04 Precise).
The bugreport seems me ok (see the end of my mail) both with the option -s
and -m.
My msnoise test also works apparently fine (output of my test as attach).
My problem is with msnoise install. I obtain a list of unclear messages (at
least for me) with no option for selecting sqlite or mysql (see also the
end of my mail)
Any hints are welcome!
Thanks for your work on this package
and my best regards
Giuseppe Di Giulio
*************************************
My bugreport
(snakes)dati@salomon-OptiPlex-9010:~/P$ sudo msnoise bugreport -s
Let's Bug Report MSNoise !
************* Computer Report *************
----------------+SYSTEM+-------------------
Linux
salomon-OptiPlex-9010
3.2.0-45-generic
#70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013
x86_64
x86_64
debian - wheezy/sid -
----------------+PYTHON+-------------------
Python:3.5.1 |Continuum Analytics, Inc.| (default, Dec 7 2015, 11:16:01)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
****************************************
************************************
My msnoise install
(snakes)dati@salomon-OptiPlex-9010:~/P$ sudo msnoise install
Launching the installer
Welcome to MSNoise
What database technology do you want to use?
[1] sqlite
[2] mysql
Traceback (most recent call last):
File "/home/dati/anaconda2/envs/snakes/bin/msnoise", line 9, in <module>
load_entry_point('msnoise==0+unknown', 'console_scripts', 'msnoise')()
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/msnoise/scripts/msnoise.py",
line 614, in run
cli(obj={})
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/click/core.py",
line 716, in __call__
return self.main(*args, **kwargs)
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/click/core.py",
line 696, in main
rv = self.invoke(ctx)
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/click/core.py",
line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/click/core.py",
line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/click/core.py",
line 534, in invoke
return callback(*args, **kwargs)
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/msnoise/scripts/msnoise.py",
line 204, in install
main()
File
"/home/dati/anaconda2/envs/snakes/lib/python3.5/site-packages/msnoise/s000installer.py",
line 48, in main
tech = int(raw_input('Choice:'))
NameError: name 'raw_input' is not defined
**********************************************+
Hi All;
I would say that before processing a certain dataset it is important to get to know your data and define what are you looking for,
or scientific question. If one knows or understands the source that is generating the random wave field and its interaction with the medium and the susceptibility to changes then it is easier to select a dataset, moving windows, frequencies, etc.
For instance, Initially small subsets centered at some specific transient event work well.
Esteban
> On May 2, 2016, at 12:44 AM, msnoise-request(a)mailman-as.oma.be wrote:
>
> Send MSNoise mailing list submissions to
> msnoise(a)mailman-as.oma.be
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman-as.oma.be/mailman/listinfo/msnoise
> or, via email, send a message with subject or body 'help' to
> msnoise-request(a)mailman-as.oma.be
>
> You can reach the person managing the list at
> msnoise-owner(a)mailman-as.oma.be
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of MSNoise digest..."
>
>
> Today's Topics:
>
> 1. Re: advice on processing database subsets (Lukas Preiswerk)
> 2. Re: advice on processing database subsets (Thomas Lecocq)
> 3. Re: advice on processing database subsets (Phil Cummins)
> 4. Re: advice on processing database subsets (Thomas Lecocq)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 1 May 2016 16:52:06 +0200
> From: Lukas Preiswerk <preiswerk(a)vaw.baug.ethz.ch>
> To: Python Package for Monitoring Seismic Velocity Changes using
> Ambient Seismic Noise <msnoise(a)mailman-as.oma.be>
> Subject: Re: [MSNoise] advice on processing database subsets
> Message-ID:
> <CAOSnoQ3rCc5U7i9TVTBMKngZfcM1gCYT7paudi9Hg=x=rCSG3g(a)mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Hi all
>
> I was in a similar situation as Phil, and I used (1). It?s not
> straightforward to copy the database and make msnoise work again in a new
> directory. But it?s definitely possible.
> I actually think it would be a nice addition to msnoise to not only make an
> option for multiple filters, but also for multiple other parameters (window
> lengths, overlaps, windsorizing, etc.). This would really help in the first
> ?exploratory phase? to find out what is the best way to process your
> dataset.
> What do you think of this idea? Practically I would implement it by moving
> these parameters (window length etc.) to the filter parameters, and treat
> it in the same way as an additional filter. As far as I understand the
> code, this wouldn?t require many adaptions?
>
> Lukas
>
>
>
> 2016-05-01 11:35 GMT+02:00 Thomas Lecocq <Thomas.Lecocq(a)seismology.be>:
>
>> Hi Phil,
>>
>> I'd say (3) would be better indeed. You can script msnoise using the api.
>> If you need to change params in the config, you can alternatively use the
>> "msnoise config --set name=value" command.
>>
>> Please keep me updated of your progresses & tests !
>>
>> Thomas
>>
>>
>>
>> On 01/05/2016 10:34, Phil Cummins wrote:
>>
>>> Hi again,
>>>
>>> As some of you may recall, I'm just getting started with msnoise. I have
>>> a large database and have managed to get my station and data availability
>>> tables populated.
>>> At this point, rather than running through the whole database, processing
>>> it with parameters I hope might work, I'd rather process small subsets,
>>> e.g. 1 day at a time, to experiment with window lengths, overlaps, etc., to
>>> find what seems optimal. My question is, what's the best way to process
>>> subsets of my database?
>>> It seems to me I have several options:
>>> (1) Make separate databases for each subset I want to test, and run
>>> through the workflow on each
>>> (2) Set start and end times appropriate for my subset, re-scan and
>>> run through the workflow.
>>> (3) Populate the jobs table, and write a script to activate only the
>>> jobs I want and not the others.
>>> I want to a fair bit of testing using different parameters before I run
>>> through the whole thing, so I think (3) may be best. But any advice would
>>> be appreciated.
>>> Regards,
>>>
>>> - Phil
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise(a)mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>
>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise(a)mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>
>
>
> ------------------------------
>
> Message: 2
> Date: Sun, 1 May 2016 20:18:26 +0200
> From: Thomas Lecocq <Thomas.Lecocq(a)seismology.be>
> To: msnoise(a)mailman-as.oma.be
> Subject: Re: [MSNoise] advice on processing database subsets
> Message-ID: <8ee5b4f1-ce82-fa13-b898-9ddd1743451e(a)seismology.be>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hi guys,
>
> Yeah, I have been thinking about a "benchmark" mode for quite a number
> of weeks, i.e. since I tested a first run of PWS in order to compare the
> final dv/v ; to compare properly I have to test quite a number of
> parameters.
>
> My current idea is to run a set of possible parameters, for different
> steps. This would lead to a large number of branches in a large tree,
> but it would definitively be quite interesting.
>
> I am really not in favor of duplicating the database, rather to create
> a "config" file with an caller script, to set/change/ parameters...
> Theoretically, the API should let you do all the actions. The only thing
> that would be a little trickier is to store/reuse the results of each
> step in order to compare them. For info, using the "shutil" module you
> can move/copy files easily.
>
> Let's keep brainstorming on that and see how it goes !
>
> Cheers
>
> Thomas
>
> On 01/05/2016 16:52, Lukas Preiswerk wrote:
>> Hi all
>>
>> I was in a similar situation as Phil, and I used (1). It?s not
>> straightforward to copy the database and make msnoise work again in a new
>> directory. But it?s definitely possible.
>> I actually think it would be a nice addition to msnoise to not only make an
>> option for multiple filters, but also for multiple other parameters (window
>> lengths, overlaps, windsorizing, etc.). This would really help in the first
>> ?exploratory phase? to find out what is the best way to process your
>> dataset.
>> What do you think of this idea? Practically I would implement it by moving
>> these parameters (window length etc.) to the filter parameters, and treat
>> it in the same way as an additional filter. As far as I understand the
>> code, this wouldn?t require many adaptions?
>>
>> Lukas
>>
>>
>>
>> 2016-05-01 11:35 GMT+02:00 Thomas Lecocq <Thomas.Lecocq(a)seismology.be>:
>>
>>> Hi Phil,
>>>
>>> I'd say (3) would be better indeed. You can script msnoise using the api.
>>> If you need to change params in the config, you can alternatively use the
>>> "msnoise config --set name=value" command.
>>>
>>> Please keep me updated of your progresses & tests !
>>>
>>> Thomas
>>>
>>>
>>>
>>> On 01/05/2016 10:34, Phil Cummins wrote:
>>>
>>>> Hi again,
>>>>
>>>> As some of you may recall, I'm just getting started with msnoise. I have
>>>> a large database and have managed to get my station and data availability
>>>> tables populated.
>>>> At this point, rather than running through the whole database, processing
>>>> it with parameters I hope might work, I'd rather process small subsets,
>>>> e.g. 1 day at a time, to experiment with window lengths, overlaps, etc., to
>>>> find what seems optimal. My question is, what's the best way to process
>>>> subsets of my database?
>>>> It seems to me I have several options:
>>>> (1) Make separate databases for each subset I want to test, and run
>>>> through the workflow on each
>>>> (2) Set start and end times appropriate for my subset, re-scan and
>>>> run through the workflow.
>>>> (3) Populate the jobs table, and write a script to activate only the
>>>> jobs I want and not the others.
>>>> I want to a fair bit of testing using different parameters before I run
>>>> through the whole thing, so I think (3) may be best. But any advice would
>>>> be appreciated.
>>>> Regards,
>>>>
>>>> - Phil
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise(a)mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise(a)mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise(a)mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>
>
>
> ------------------------------
>
> Message: 3
> Date: Mon, 2 May 2016 17:41:20 +1000
> From: Phil Cummins <phil.cummins(a)anu.edu.au>
> To: Python Package for Monitoring Seismic Velocity Changes using
> Ambient Seismic Noise <msnoise(a)mailman-as.oma.be>
> Subject: Re: [MSNoise] advice on processing database subsets
> Message-ID: <572704A0.1030608(a)anu.edu.au>
> Content-Type: text/plain; charset="UTF-8"; format=flowed
>
> Hi again,
>
> Thanks for the comments. Here's what I did to set just a singe day for
> processing, so that I can test the parameter settings. I looked into the
> API code and needed to import from msnoise_table_def.py, but it seems to
> work OK:
>
> from msnoise.api import connect
> from msnoise_table_def import Job
>
> set_day = '2013-10-14'
> jobtype = 'CC'
> session = connect()
> jobs_set = session.query(Job).filter(Job.jobtype ==
> jobtype).filter(Job.day == set_day)
> jobs_set.update({Job.flag: 'T'})
> jobs_unset = session.query(Job).filter(Job.jobtype ==
> jobtype).filter(Job.day != set_day)
> jobs_unset.update({Job.flag: 'D'})
> session.commit()
>
> So now I have a jobs table with just the day I want set to 'T'. I hoped
> I was ready to try 'msnoise compute_cc', but it seems to want me to set
> Filters first. This appears to be referring to the MCWS filter
> parameters? I am a little surprised since I thought MCWS would come
> later, and don't understand how the CC computation would be dependent on
> the MCWS filter parameters.
>
> To tell you the truth, at the moment I am more interested in using the
> msnoise cross-correlations as input to a tomography algorithm, rather
> than in MCWS itself. In any case I am keen to look at the CC to see that
> they make sense, before i move to anything else.
>
> Would it be possible to please advise on whether there is a way to do
> compute_cc without having to worry about the MCWS parameters?
>
> Thanks,
>
> - Phil
>
>
> Thomas Lecocq wrote:
>> Hi guys,
>>
>> Yeah, I have been thinking about a "benchmark" mode for quite a number
>> of weeks, i.e. since I tested a first run of PWS in order to compare
>> the final dv/v ; to compare properly I have to test quite a number of
>> parameters.
>>
>> My current idea is to run a set of possible parameters, for different
>> steps. This would lead to a large number of branches in a large tree,
>> but it would definitively be quite interesting.
>>
>> I am really not in favor of duplicating the database, rather to
>> create a "config" file with an caller script, to set/change/
>> parameters... Theoretically, the API should let you do all the
>> actions. The only thing that would be a little trickier is to
>> store/reuse the results of each step in order to compare them. For
>> info, using the "shutil" module you can move/copy files easily.
>>
>> Let's keep brainstorming on that and see how it goes !
>>
>> Cheers
>>
>> Thomas
>>
>> On 01/05/2016 16:52, Lukas Preiswerk wrote:
>>> Hi all
>>>
>>> I was in a similar situation as Phil, and I used (1). It?s not
>>> straightforward to copy the database and make msnoise work again in a
>>> new
>>> directory. But it?s definitely possible.
>>> I actually think it would be a nice addition to msnoise to not only
>>> make an
>>> option for multiple filters, but also for multiple other parameters
>>> (window
>>> lengths, overlaps, windsorizing, etc.). This would really help in the
>>> first
>>> ?exploratory phase? to find out what is the best way to process your
>>> dataset.
>>> What do you think of this idea? Practically I would implement it by
>>> moving
>>> these parameters (window length etc.) to the filter parameters, and
>>> treat
>>> it in the same way as an additional filter. As far as I understand the
>>> code, this wouldn?t require many adaptions?
>>>
>>> Lukas
>>>
>>>
>>>
>>> 2016-05-01 11:35 GMT+02:00 Thomas Lecocq <Thomas.Lecocq(a)seismology.be>:
>>>
>>>> Hi Phil,
>>>>
>>>> I'd say (3) would be better indeed. You can script msnoise using the
>>>> api.
>>>> If you need to change params in the config, you can alternatively
>>>> use the
>>>> "msnoise config --set name=value" command.
>>>>
>>>> Please keep me updated of your progresses & tests !
>>>>
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On 01/05/2016 10:34, Phil Cummins wrote:
>>>>
>>>>> Hi again,
>>>>>
>>>>> As some of you may recall, I'm just getting started with msnoise. I
>>>>> have
>>>>> a large database and have managed to get my station and data
>>>>> availability
>>>>> tables populated.
>>>>> At this point, rather than running through the whole database,
>>>>> processing
>>>>> it with parameters I hope might work, I'd rather process small
>>>>> subsets,
>>>>> e.g. 1 day at a time, to experiment with window lengths, overlaps,
>>>>> etc., to
>>>>> find what seems optimal. My question is, what's the best way to
>>>>> process
>>>>> subsets of my database?
>>>>> It seems to me I have several options:
>>>>> (1) Make separate databases for each subset I want to test,
>>>>> and run
>>>>> through the workflow on each
>>>>> (2) Set start and end times appropriate for my subset, re-scan
>>>>> and
>>>>> run through the workflow.
>>>>> (3) Populate the jobs table, and write a script to activate
>>>>> only the
>>>>> jobs I want and not the others.
>>>>> I want to a fair bit of testing using different parameters before I
>>>>> run
>>>>> through the whole thing, so I think (3) may be best. But any advice
>>>>> would
>>>>> be appreciated.
>>>>> Regards,
>>>>>
>>>>> - Phil
>>>>> _______________________________________________
>>>>> MSNoise mailing list
>>>>> MSNoise(a)mailman-as.oma.be
>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>>
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise(a)mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise(a)mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise(a)mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 2 May 2016 09:44:38 +0200
> From: Thomas Lecocq <thomas.lecocq(a)oma.be>
> To: msnoise(a)mailman-as.oma.be
> Subject: Re: [MSNoise] advice on processing database subsets
> Message-ID: <57270566.40602(a)oma.be>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hi Phil,
>
> Nice piece of code.
>
> Currently, a Filter is defined for AND cc step (whitening) AND mwcs
> step. So, you'll have to define the filter's bounds for the CC step,
> while keeping the MWCS values to 0 , e.g. setting "low", "high",
> "rms_threshold=0", "used=true" and you'll be good to go !
>
> Thomas
>
> Le 02/05/2016 09:41, Phil Cummins a ?crit :
>> Hi again,
>>
>> Thanks for the comments. Here's what I did to set just a singe day for
>> processing, so that I can test the parameter settings. I looked into
>> the API code and needed to import from msnoise_table_def.py, but it
>> seems to work OK:
>>
>> from msnoise.api import connect
>> from msnoise_table_def import Job
>>
>> set_day = '2013-10-14'
>> jobtype = 'CC'
>> session = connect()
>> jobs_set = session.query(Job).filter(Job.jobtype ==
>> jobtype).filter(Job.day == set_day)
>> jobs_set.update({Job.flag: 'T'})
>> jobs_unset = session.query(Job).filter(Job.jobtype ==
>> jobtype).filter(Job.day != set_day)
>> jobs_unset.update({Job.flag: 'D'})
>> session.commit()
>>
>> So now I have a jobs table with just the day I want set to 'T'. I
>> hoped I was ready to try 'msnoise compute_cc', but it seems to want me
>> to set Filters first. This appears to be referring to the MCWS filter
>> parameters? I am a little surprised since I thought MCWS would come
>> later, and don't understand how the CC computation would be dependent
>> on the MCWS filter parameters.
>>
>> To tell you the truth, at the moment I am more interested in using the
>> msnoise cross-correlations as input to a tomography algorithm, rather
>> than in MCWS itself. In any case I am keen to look at the CC to see
>> that they make sense, before i move to anything else.
>>
>> Would it be possible to please advise on whether there is a way to do
>> compute_cc without having to worry about the MCWS parameters?
>>
>> Thanks,
>>
>> - Phil
>>
>>
>> Thomas Lecocq wrote:
>>> Hi guys,
>>>
>>> Yeah, I have been thinking about a "benchmark" mode for quite a
>>> number of weeks, i.e. since I tested a first run of PWS in order to
>>> compare the final dv/v ; to compare properly I have to test quite a
>>> number of parameters.
>>>
>>> My current idea is to run a set of possible parameters, for different
>>> steps. This would lead to a large number of branches in a large tree,
>>> but it would definitively be quite interesting.
>>>
>>> I am really not in favor of duplicating the database, rather to
>>> create a "config" file with an caller script, to set/change/
>>> parameters... Theoretically, the API should let you do all the
>>> actions. The only thing that would be a little trickier is to
>>> store/reuse the results of each step in order to compare them. For
>>> info, using the "shutil" module you can move/copy files easily.
>>>
>>> Let's keep brainstorming on that and see how it goes !
>>>
>>> Cheers
>>>
>>> Thomas
>>>
>>> On 01/05/2016 16:52, Lukas Preiswerk wrote:
>>>> Hi all
>>>>
>>>> I was in a similar situation as Phil, and I used (1). It?s not
>>>> straightforward to copy the database and make msnoise work again in
>>>> a new
>>>> directory. But it?s definitely possible.
>>>> I actually think it would be a nice addition to msnoise to not only
>>>> make an
>>>> option for multiple filters, but also for multiple other parameters
>>>> (window
>>>> lengths, overlaps, windsorizing, etc.). This would really help in
>>>> the first
>>>> ?exploratory phase? to find out what is the best way to process your
>>>> dataset.
>>>> What do you think of this idea? Practically I would implement it by
>>>> moving
>>>> these parameters (window length etc.) to the filter parameters, and
>>>> treat
>>>> it in the same way as an additional filter. As far as I understand the
>>>> code, this wouldn?t require many adaptions?
>>>>
>>>> Lukas
>>>>
>>>>
>>>>
>>>> 2016-05-01 11:35 GMT+02:00 Thomas Lecocq <Thomas.Lecocq(a)seismology.be>:
>>>>
>>>>> Hi Phil,
>>>>>
>>>>> I'd say (3) would be better indeed. You can script msnoise using
>>>>> the api.
>>>>> If you need to change params in the config, you can alternatively
>>>>> use the
>>>>> "msnoise config --set name=value" command.
>>>>>
>>>>> Please keep me updated of your progresses & tests !
>>>>>
>>>>> Thomas
>>>>>
>>>>>
>>>>>
>>>>> On 01/05/2016 10:34, Phil Cummins wrote:
>>>>>
>>>>>> Hi again,
>>>>>>
>>>>>> As some of you may recall, I'm just getting started with msnoise.
>>>>>> I have
>>>>>> a large database and have managed to get my station and data
>>>>>> availability
>>>>>> tables populated.
>>>>>> At this point, rather than running through the whole database,
>>>>>> processing
>>>>>> it with parameters I hope might work, I'd rather process small
>>>>>> subsets,
>>>>>> e.g. 1 day at a time, to experiment with window lengths, overlaps,
>>>>>> etc., to
>>>>>> find what seems optimal. My question is, what's the best way to
>>>>>> process
>>>>>> subsets of my database?
>>>>>> It seems to me I have several options:
>>>>>> (1) Make separate databases for each subset I want to test,
>>>>>> and run
>>>>>> through the workflow on each
>>>>>> (2) Set start and end times appropriate for my subset,
>>>>>> re-scan and
>>>>>> run through the workflow.
>>>>>> (3) Populate the jobs table, and write a script to activate
>>>>>> only the
>>>>>> jobs I want and not the others.
>>>>>> I want to a fair bit of testing using different parameters before
>>>>>> I run
>>>>>> through the whole thing, so I think (3) may be best. But any
>>>>>> advice would
>>>>>> be appreciated.
>>>>>> Regards,
>>>>>>
>>>>>> - Phil
>>>>>> _______________________________________________
>>>>>> MSNoise mailing list
>>>>>> MSNoise(a)mailman-as.oma.be
>>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>>>
>>>>> _______________________________________________
>>>>> MSNoise mailing list
>>>>> MSNoise(a)mailman-as.oma.be
>>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>>>
>>>> _______________________________________________
>>>> MSNoise mailing list
>>>> MSNoise(a)mailman-as.oma.be
>>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>>>
>>> _______________________________________________
>>> MSNoise mailing list
>>> MSNoise(a)mailman-as.oma.be
>>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>> _______________________________________________
>> MSNoise mailing list
>> MSNoise(a)mailman-as.oma.be
>> http://mailman-as.oma.be/mailman/listinfo/msnoise
>
>
>
> ------------------------------
>
> _______________________________________________
> MSNoise mailing list
> MSNoise(a)mailman-as.oma.be
> http://mailman-as.oma.be/mailman/listinfo/msnoise
>
>
> End of MSNoise Digest, Vol 27, Issue 2
> **************************************
Hi again,
As some of you may recall, I'm just getting started with msnoise. I have
a large database and have managed to get my station and data
availability tables populated.
At this point, rather than running through the whole database,
processing it with parameters I hope might work, I'd rather process
small subsets, e.g. 1 day at a time, to experiment with window lengths,
overlaps, etc., to find what seems optimal. My question is, what's the
best way to process subsets of my database?
It seems to me I have several options:
(1) Make separate databases for each subset I want to test, and run
through the workflow on each
(2) Set start and end times appropriate for my subset, re-scan and
run through the workflow.
(3) Populate the jobs table, and write a script to activate only
the jobs I want and not the others.
I want to a fair bit of testing using different parameters before I run
through the whole thing, so I think (3) may be best. But any advice
would be appreciated.
Regards,
- Phil
