Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings
Discussion options

tanmayGIT
Aug 24, 2022

Hi, I am performing an experiment to measure STUMP's computational time, which uses STOMP_Opt at it's origin.
I have a 1D time series (even tried with 4 different time series) of 100k length and am increasing the subsequence length by 2000 in each iteration to see the change in computational time.

Ideally, the computational time should decrease but it is kind of fluctuating. Another point is why always the computational time for the very first iteration is very high ?
I suspect that it may happen due to the use of threads and the thread synchronization issues. But whether there are any concise details about the reason for it. I need to write in my article where the reviewer has asked to compare the performance by using the STUMPY library.

Here is one example:

The sub-sequence length 2000 and the needed time is 19.032682 ??
The sub-sequence length 4000 and the needed time is 10.091411
The sub-sequence length 6000 and the needed time is 10.302999
The sub-sequence length 8000 and the needed time is 10.007654
The sub-sequence length 10000 and the needed time is 9.512854
The sub-sequence length 12000 and the needed time is 20.557093
The sub-sequence length 14000 and the needed time is 17.757711
The sub-sequence length 16000 and the needed time is 17.917705
The sub-sequence length 18000 and the needed time is 17.200630
The sub-sequence length 20000 and the needed time is 15.299617
The sub-sequence length 22000 and the needed time is 15.026888
The sub-sequence length 24000 and the needed time is 14.469026
The sub-sequence length 26000 and the needed time is 13.799839
The sub-sequence length 28000 and the needed time is 12.855113

Thanks in advance for your help and more insights.
You must be logged in to vote

Replies: 2 comments · 1 reply

Comment options

seanlaw
Aug 24, 2022
Maintainer

@tanmayGIT Thank you for your question and welcome to the STUMPY community. Performing timing comparisons is complex as there are many, many factors that may affect the timing behind the scenes.

why always the computational time for the very first iteration is very high ?

This is because STUMPY use numba, which JIT-compiles Python code into performant machine code so that it can run fast. This means that the first time you call any STUMPY function (e.g., stumpy.stump), it must spend a little bit of time to compile the Python code. However, after the first time (within the current Python session), it caches the function for repeated use. So, for any timing calculation, please ignore the first execution time as it will likely contain compilation time.

I suspect that it may happen due to the use of threads and the thread synchronization issues. But whether there are any concise details about the reason for it. I need to write in my article where the reviewer has asked to compare the performance by using the STUMPY library.

As stated above, timing calculations are really hard to get right and the problem is that there may be background processes running on your machine that may be taking up memory or competing for threads. I ran some tests with:

import numpy as np
import stumpy
import time

T = np.random.rand(100_000)

stumpy.stump(T, m)  # Ignore first execution

for m in range(2000, 30000, 2000):
    elapse = []
    for _ in range(10):
        start = time.time()
        stumpy.stump(T, m)
        elapse.append(time.time()-start)
    print(m, np.mean(elapse))

and got:

2000 5.9589533567428585
4000 6.536279487609863
6000 7.243327665328979
8000 6.702517819404602
10000 6.869658279418945
12000 6.561953926086426
14000 6.685491895675659
16000 6.94999737739563
18000 7.2453858137130736
20000 7.042616391181946
22000 6.986396670341492
24000 6.718616390228272
26000 6.675270819664002
28000 6.444169235229492

I think this is about as good/close as you're going to get. To be 100% clear, I do not believe that this is a problem with STUMPY! Unless you can eliminate all possible background processes (including those executed/managed by your kernel) and clear out the memory being used, I don't think this timing comparison is meaningful.
You must be logged in to vote
0 replies
Comment options

damianzd1
Aug 16, 2023

I've been trying but without success to cache stumpy.stump function so I can reuse it between scripts. In my particular case I am trying to build a docker container for AWS Lambda to be used as AWS API Gateway endpoint. The problem is that compiling stumpy.stump takes more time that AWS API Gateway allows a request to hang, so the call always fails. Can it be cached and reused between python sessions?
You must be logged in to vote
1 reply
@seanlaw
Comment options

seanlaw Aug 16, 2023
Maintainer

@damianzd1 For ease for searching, I would like to move this question to its own independent discussion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
❓
Q&A
Labels
None yet
3 participants
@tanmayGIT @seanlaw @damianzd1
Converted from issue

This discussion was converted from issue #660 on August 24, 2022 11:31.

Morty Proxy This is a proxified and sanitized view of the page, visit original site.