Grokbase Groups R r-help June 2016
FAQ
Hello,
I have multiple text files with the format shown below (see the two files
that I pasted below). Each file is a log of multiple steps that the system
has processed and for each step, it has shown the start time of the process
step. For example, in the data below, the filter started at
06/16/2016|03:44:16

How to read this data so that Step 001 is one data frame, Step 002 is
another, and so on. After I do this, I will then compare the Step 001 times
with and without parallel process.


For example, the files pasted below "no_parallel_process_SLS_4.txt" and
"parallel_process_SLS_4.txt" will make it clear what I am trying to do. I
want to compare the parallel process times taken for each step with the non
parallel process times.


If there are better ways of performing this task that what I am thinking,
could you let me know? Thanks in advance.


Satish Vadlamani

parallel_process_file.txt
06/16/2016|03:44:16|Step 001
06/16/2016|03:44:16|Initialization
06/16/2016|03:44:16|Filters
06/16/2016|03:45:03|Split Items
06/16/2016|03:46:20|Sort
06/16/2016|03:46:43|Check
06/16/2016|04:01:13|Save
06/16/2016|04:04:35|Update preparation
06/16/2016|04:04:36|Update comparison
06/16/2016|04:04:38|Update
06/16/2016|04:04:38|Update
06/16/2016|04:06:01|Close
06/16/2016|04:06:33|BOP processing for 7,960 items has finished
06/16/2016|04:06:34|Step 002
06/16/2016|04:06:35|Initialization
06/16/2016|04:06:35|Filters
06/16/2016|04:07:14|Split Items
06/16/2016|04:08:57|Sort
06/16/2016|04:09:06|Check
06/16/2016|04:26:36|Save
06/16/2016|04:39:29|Update preparation
06/16/2016|04:39:31|Update comparison
06/16/2016|04:39:43|Update
06/16/2016|04:39:45|Update
06/16/2016|04:44:28|Close
06/16/2016|04:45:26|BOP processing for 8,420 items has finished
06/16/2016|04:45:27|Step 003
06/16/2016|04:45:27|Initialization
06/16/2016|04:45:27|Filters
06/16/2016|04:48:50|Split Items
06/16/2016|04:55:15|Sort
06/16/2016|04:55:40|Check
06/16/2016|05:13:35|Save
06/16/2016|05:17:34|Update preparation
06/16/2016|05:17:34|Update comparison
06/16/2016|05:17:36|Update
06/16/2016|05:17:36|Update
06/16/2016|05:19:29|Close
06/16/2016|05:19:49|BOP processing for 8,876 items has finished
06/16/2016|05:19:50|Step 004
06/16/2016|05:19:50|Initialization
06/16/2016|05:19:50|Filters
06/16/2016|05:20:43|Split Items
06/16/2016|05:22:14|Sort
06/16/2016|05:22:29|Check
06/16/2016|05:37:27|Save
06/16/2016|05:38:43|Update preparation
06/16/2016|05:38:44|Update comparison
06/16/2016|05:38:45|Update
06/16/2016|05:38:45|Update
06/16/2016|05:39:09|Close
06/16/2016|05:39:19|BOP processing for 5,391 items has finished
06/16/2016|05:39:20|Step 005
06/16/2016|05:39:20|Initialization
06/16/2016|05:39:20|Filters
06/16/2016|05:39:57|Split Items
06/16/2016|05:40:21|Sort
06/16/2016|05:40:24|Check
06/16/2016|05:46:01|Save
06/16/2016|05:46:54|Update preparation
06/16/2016|05:46:54|Update comparison
06/16/2016|05:46:54|Update
06/16/2016|05:46:55|Update
06/16/2016|05:47:24|Close
06/16/2016|05:47:31|BOP processing for 3,016 items has finished
06/16/2016|05:47:32|Step 006
06/16/2016|05:47:32|Initialization
06/16/2016|05:47:32|Filters
06/16/2016|05:47:32|Update preparation
06/16/2016|05:47:32|Update comparison
06/16/2016|05:47:32|Update
06/16/2016|05:47:32|Close
06/16/2016|05:47:33|BOP processing for 0 items has finished
06/16/2016|05:47:33|Step 007
06/16/2016|05:47:33|Initialization
06/16/2016|05:47:33|Filters
06/16/2016|05:47:34|Split Items
06/16/2016|05:47:34|Sort
06/16/2016|05:47:34|Check
06/16/2016|05:47:37|Save
06/16/2016|05:47:37|Update preparation
06/16/2016|05:47:37|Update comparison
06/16/2016|05:47:37|Update
06/16/2016|05:47:37|Update
06/16/2016|05:47:37|Close
06/16/2016|05:47:37|BOP processing for 9 items has finished
06/16/2016|05:47:37|Step 008
06/16/2016|05:47:37|Initialization
06/16/2016|05:47:37|Filters
06/16/2016|05:47:38|Update preparation
06/16/2016|05:47:38|Update comparison
06/16/2016|05:47:38|Update
06/16/2016|05:47:38|Close
06/16/2016|05:47:38|BOP processing for 0 items has finished






no_parallel_process_file.txt
06/15/2016|22:52:46|Step 001
06/15/2016|22:52:46|Initialization
06/15/2016|22:52:46|Filters
06/15/2016|22:54:21|Split Items
06/15/2016|22:55:10|Sort
06/15/2016|22:55:15|Check
06/15/2016|23:04:43|Save
06/15/2016|23:06:38|Update preparation
06/15/2016|23:06:38|Update comparison
06/15/2016|23:06:39|Update
06/15/2016|23:06:39|Update
06/15/2016|23:12:04|Close
06/15/2016|23:13:16|BOP processing for 7,942 items has finished
06/15/2016|23:13:17|Step 002
06/15/2016|23:13:17|Initialization
06/15/2016|23:13:17|Filters
06/15/2016|23:16:27|Split Items
06/15/2016|23:20:18|Sort
06/15/2016|23:20:34|Check
06/16/2016|00:08:08|Save
06/16/2016|00:26:19|Update preparation
06/16/2016|00:26:20|Update comparison
06/16/2016|00:26:30|Update
06/16/2016|00:26:31|Update
06/16/2016|00:42:31|Close
06/16/2016|00:45:09|BOP processing for 8,400 items has finished
06/16/2016|00:45:11|Step 003
06/16/2016|00:45:12|Initialization
06/16/2016|00:45:12|Filters
06/16/2016|00:53:01|Split Items
06/16/2016|01:01:44|Sort
06/16/2016|01:02:55|Check
06/16/2016|01:41:40|Save
06/16/2016|01:44:37|Update preparation
06/16/2016|01:44:37|Update comparison
06/16/2016|01:44:39|Update
06/16/2016|01:44:39|Update
06/16/2016|01:47:37|Close
06/16/2016|01:48:07|BOP processing for 8,867 items has finished
06/16/2016|01:48:08|Step 004
06/16/2016|01:48:08|Initialization
06/16/2016|01:48:08|Filters
06/16/2016|01:49:51|Split Items
06/16/2016|01:50:35|Sort
06/16/2016|01:50:39|Check
06/16/2016|01:59:12|Save
06/16/2016|02:00:47|Update preparation
06/16/2016|02:00:47|Update comparison
06/16/2016|02:00:48|Update
06/16/2016|02:00:48|Update
06/16/2016|02:02:40|Close
06/16/2016|02:02:55|BOP processing for 5,383 items has finished
06/16/2016|02:02:56|Step 005
06/16/2016|02:02:56|Initialization
06/16/2016|02:02:56|Filters
06/16/2016|02:03:47|Split Items
06/16/2016|02:04:19|Sort
06/16/2016|02:04:21|Check
06/16/2016|02:08:08|Save
06/16/2016|02:09:22|Update preparation
06/16/2016|02:09:22|Update comparison
06/16/2016|02:09:22|Update
06/16/2016|02:09:22|Update
06/16/2016|02:11:03|Close
06/16/2016|02:11:14|BOP processing for 3,016 items has finished
06/16/2016|02:11:14|Step 006
06/16/2016|02:11:14|Initialization
06/16/2016|02:11:14|Filters
06/16/2016|02:11:15|Update preparation
06/16/2016|02:11:15|Update comparison
06/16/2016|02:11:15|Update
06/16/2016|02:11:15|Close
06/16/2016|02:11:15|BOP processing for 0 items has finished
06/16/2016|02:11:15|Step 007
06/16/2016|02:11:15|Initialization
06/16/2016|02:11:15|Filters
06/16/2016|02:11:17|Split Items
06/16/2016|02:11:17|Sort
06/16/2016|02:11:17|Check
06/16/2016|02:11:20|Save
06/16/2016|02:11:20|Update preparation
06/16/2016|02:11:20|Update comparison
06/16/2016|02:11:20|Update
06/16/2016|02:11:20|Update
06/16/2016|02:11:20|Close
06/16/2016|02:11:20|BOP processing for 9 items has finished
06/16/2016|02:11:20|Step 008
06/16/2016|02:11:20|Initialization
06/16/2016|02:11:21|Filters
06/16/2016|02:11:21|Update preparation
06/16/2016|02:11:21|Update comparison
06/16/2016|02:11:21|Update
06/16/2016|02:11:21|Close
06/16/2016|02:11:21|BOP processing for 0 items has finished





--


Satish Vadlamani


  [[alternative HTML version deleted]]

Search Discussions

  • William Dunlap at Jun 17, 2016 at 3:57 pm
    You can make a step-number variable with cumsum(grepl("^Step ", ...)) and
    use it as the splitting variable in split. E.g.,

    dat <- read.table(yourFile, stringsAsFactorsúLSE, sep="|",
    colClasses=c("NULL", "character", "character", "character"),
    col.names=c("Junk","Date","Time","Type"))
    dat <- with(dat, data.frame(DateTime=as.POSIXct(paste(Date, Time),
    format="%m/%d/%Y %H:%M:%S"), Type=Type, stringsAsFactorsúLSE))
    head(dat)
                  DateTime Type
    1 2016-06-16 03:44:16 Step 001
    2 2016-06-16 03:44:16 Initialization
    3 2016-06-16 03:44:16 Filters
    4 2016-06-16 03:45:03 Split Items
    5 2016-06-16 03:46:20 Sort
    6 2016-06-16 03:46:43 Check
    split(dat, cumsum(grepl("^Step ", dat$Type)))
    $`1`
                   DateTime Type
    1 2016-06-16 03:44:16 Step 001
    2 2016-06-16 03:44:16 Initialization
    ...
    13 2016-06-16 04:06:33 BOP processing for 7,960 items has finished


    $`2`
                   DateTime Type
    14 2016-06-16 04:06:34 Step 002
    15 2016-06-16 04:06:35 Initialization
    ...






    Bill Dunlap
    TIBCO Software
    wdunlap tibco.com


    On Thu, Jun 16, 2016 at 8:42 PM, Satish Vadlamani wrote:

    Hello,
    I have multiple text files with the format shown below (see the two files
    that I pasted below). Each file is a log of multiple steps that the system
    has processed and for each step, it has shown the start time of the process
    step. For example, in the data below, the filter started at
    06/16/2016|03:44:16
    How to read this data so that Step 001 is one data frame, Step 002 is
    another, and so on. After I do this, I will then compare the Step 001 times
    with and without parallel process.

    For example, the files pasted below "no_parallel_process_SLS_4.txt" and
    "parallel_process_SLS_4.txt" will make it clear what I am trying to do. I
    want to compare the parallel process times taken for each step with the non
    parallel process times.

    If there are better ways of performing this task that what I am thinking,
    could you let me know? Thanks in advance.

    Satish Vadlamani
    parallel_process_file.txt
    06/16/2016|03:44:16|Step 001
    06/16/2016|03:44:16|Initialization
    06/16/2016|03:44:16|Filters
    06/16/2016|03:45:03|Split Items
    06/16/2016|03:46:20|Sort
    06/16/2016|03:46:43|Check
    06/16/2016|04:01:13|Save
    06/16/2016|04:04:35|Update preparation
    06/16/2016|04:04:36|Update comparison
    06/16/2016|04:04:38|Update
    06/16/2016|04:04:38|Update
    06/16/2016|04:06:01|Close
    06/16/2016|04:06:33|BOP processing for 7,960 items has finished
    06/16/2016|04:06:34|Step 002
    06/16/2016|04:06:35|Initialization
    06/16/2016|04:06:35|Filters
    06/16/2016|04:07:14|Split Items
    06/16/2016|04:08:57|Sort
    06/16/2016|04:09:06|Check
    06/16/2016|04:26:36|Save
    06/16/2016|04:39:29|Update preparation
    06/16/2016|04:39:31|Update comparison
    06/16/2016|04:39:43|Update
    06/16/2016|04:39:45|Update
    06/16/2016|04:44:28|Close
    06/16/2016|04:45:26|BOP processing for 8,420 items has finished
    06/16/2016|04:45:27|Step 003
    06/16/2016|04:45:27|Initialization
    06/16/2016|04:45:27|Filters
    06/16/2016|04:48:50|Split Items
    06/16/2016|04:55:15|Sort
    06/16/2016|04:55:40|Check
    06/16/2016|05:13:35|Save
    06/16/2016|05:17:34|Update preparation
    06/16/2016|05:17:34|Update comparison
    06/16/2016|05:17:36|Update
    06/16/2016|05:17:36|Update
    06/16/2016|05:19:29|Close
    06/16/2016|05:19:49|BOP processing for 8,876 items has finished
    06/16/2016|05:19:50|Step 004
    06/16/2016|05:19:50|Initialization
    06/16/2016|05:19:50|Filters
    06/16/2016|05:20:43|Split Items
    06/16/2016|05:22:14|Sort
    06/16/2016|05:22:29|Check
    06/16/2016|05:37:27|Save
    06/16/2016|05:38:43|Update preparation
    06/16/2016|05:38:44|Update comparison
    06/16/2016|05:38:45|Update
    06/16/2016|05:38:45|Update
    06/16/2016|05:39:09|Close
    06/16/2016|05:39:19|BOP processing for 5,391 items has finished
    06/16/2016|05:39:20|Step 005
    06/16/2016|05:39:20|Initialization
    06/16/2016|05:39:20|Filters
    06/16/2016|05:39:57|Split Items
    06/16/2016|05:40:21|Sort
    06/16/2016|05:40:24|Check
    06/16/2016|05:46:01|Save
    06/16/2016|05:46:54|Update preparation
    06/16/2016|05:46:54|Update comparison
    06/16/2016|05:46:54|Update
    06/16/2016|05:46:55|Update
    06/16/2016|05:47:24|Close
    06/16/2016|05:47:31|BOP processing for 3,016 items has finished
    06/16/2016|05:47:32|Step 006
    06/16/2016|05:47:32|Initialization
    06/16/2016|05:47:32|Filters
    06/16/2016|05:47:32|Update preparation
    06/16/2016|05:47:32|Update comparison
    06/16/2016|05:47:32|Update
    06/16/2016|05:47:32|Close
    06/16/2016|05:47:33|BOP processing for 0 items has finished
    06/16/2016|05:47:33|Step 007
    06/16/2016|05:47:33|Initialization
    06/16/2016|05:47:33|Filters
    06/16/2016|05:47:34|Split Items
    06/16/2016|05:47:34|Sort
    06/16/2016|05:47:34|Check
    06/16/2016|05:47:37|Save
    06/16/2016|05:47:37|Update preparation
    06/16/2016|05:47:37|Update comparison
    06/16/2016|05:47:37|Update
    06/16/2016|05:47:37|Update
    06/16/2016|05:47:37|Close
    06/16/2016|05:47:37|BOP processing for 9 items has finished
    06/16/2016|05:47:37|Step 008
    06/16/2016|05:47:37|Initialization
    06/16/2016|05:47:37|Filters
    06/16/2016|05:47:38|Update preparation
    06/16/2016|05:47:38|Update comparison
    06/16/2016|05:47:38|Update
    06/16/2016|05:47:38|Close
    06/16/2016|05:47:38|BOP processing for 0 items has finished


    no_parallel_process_file.txt
    06/15/2016|22:52:46|Step 001
    06/15/2016|22:52:46|Initialization
    06/15/2016|22:52:46|Filters
    06/15/2016|22:54:21|Split Items
    06/15/2016|22:55:10|Sort
    06/15/2016|22:55:15|Check
    06/15/2016|23:04:43|Save
    06/15/2016|23:06:38|Update preparation
    06/15/2016|23:06:38|Update comparison
    06/15/2016|23:06:39|Update
    06/15/2016|23:06:39|Update
    06/15/2016|23:12:04|Close
    06/15/2016|23:13:16|BOP processing for 7,942 items has finished
    06/15/2016|23:13:17|Step 002
    06/15/2016|23:13:17|Initialization
    06/15/2016|23:13:17|Filters
    06/15/2016|23:16:27|Split Items
    06/15/2016|23:20:18|Sort
    06/15/2016|23:20:34|Check
    06/16/2016|00:08:08|Save
    06/16/2016|00:26:19|Update preparation
    06/16/2016|00:26:20|Update comparison
    06/16/2016|00:26:30|Update
    06/16/2016|00:26:31|Update
    06/16/2016|00:42:31|Close
    06/16/2016|00:45:09|BOP processing for 8,400 items has finished
    06/16/2016|00:45:11|Step 003
    06/16/2016|00:45:12|Initialization
    06/16/2016|00:45:12|Filters
    06/16/2016|00:53:01|Split Items
    06/16/2016|01:01:44|Sort
    06/16/2016|01:02:55|Check
    06/16/2016|01:41:40|Save
    06/16/2016|01:44:37|Update preparation
    06/16/2016|01:44:37|Update comparison
    06/16/2016|01:44:39|Update
    06/16/2016|01:44:39|Update
    06/16/2016|01:47:37|Close
    06/16/2016|01:48:07|BOP processing for 8,867 items has finished
    06/16/2016|01:48:08|Step 004
    06/16/2016|01:48:08|Initialization
    06/16/2016|01:48:08|Filters
    06/16/2016|01:49:51|Split Items
    06/16/2016|01:50:35|Sort
    06/16/2016|01:50:39|Check
    06/16/2016|01:59:12|Save
    06/16/2016|02:00:47|Update preparation
    06/16/2016|02:00:47|Update comparison
    06/16/2016|02:00:48|Update
    06/16/2016|02:00:48|Update
    06/16/2016|02:02:40|Close
    06/16/2016|02:02:55|BOP processing for 5,383 items has finished
    06/16/2016|02:02:56|Step 005
    06/16/2016|02:02:56|Initialization
    06/16/2016|02:02:56|Filters
    06/16/2016|02:03:47|Split Items
    06/16/2016|02:04:19|Sort
    06/16/2016|02:04:21|Check
    06/16/2016|02:08:08|Save
    06/16/2016|02:09:22|Update preparation
    06/16/2016|02:09:22|Update comparison
    06/16/2016|02:09:22|Update
    06/16/2016|02:09:22|Update
    06/16/2016|02:11:03|Close
    06/16/2016|02:11:14|BOP processing for 3,016 items has finished
    06/16/2016|02:11:14|Step 006
    06/16/2016|02:11:14|Initialization
    06/16/2016|02:11:14|Filters
    06/16/2016|02:11:15|Update preparation
    06/16/2016|02:11:15|Update comparison
    06/16/2016|02:11:15|Update
    06/16/2016|02:11:15|Close
    06/16/2016|02:11:15|BOP processing for 0 items has finished
    06/16/2016|02:11:15|Step 007
    06/16/2016|02:11:15|Initialization
    06/16/2016|02:11:15|Filters
    06/16/2016|02:11:17|Split Items
    06/16/2016|02:11:17|Sort
    06/16/2016|02:11:17|Check
    06/16/2016|02:11:20|Save
    06/16/2016|02:11:20|Update preparation
    06/16/2016|02:11:20|Update comparison
    06/16/2016|02:11:20|Update
    06/16/2016|02:11:20|Update
    06/16/2016|02:11:20|Close
    06/16/2016|02:11:20|BOP processing for 9 items has finished
    06/16/2016|02:11:20|Step 008
    06/16/2016|02:11:20|Initialization
    06/16/2016|02:11:21|Filters
    06/16/2016|02:11:21|Update preparation
    06/16/2016|02:11:21|Update comparison
    06/16/2016|02:11:21|Update
    06/16/2016|02:11:21|Close
    06/16/2016|02:11:21|BOP processing for 0 items has finished


    --

    Satish Vadlamani

    [[alternative HTML version deleted]]

    ______________________________________________
    R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
    https://stat.ethz.ch/mailman/listinfo/r-help
    PLEASE do read the posting guide
    http://www.R-project.org/posting-guide.html
    and provide commented, minimal, self-contained, reproducible code.

      [[alternative HTML version deleted]]

Related Discussions

Discussion Navigation
viewthread | post
Discussion Overview
groupr-help @
categoriesr
postedJun 17, '16 at 3:42a
activeJun 17, '16 at 3:57p
posts2
users2
websiter-project.org
irc#r

People

Translate

site design / logo © 2017 Grokbase