Converter data em formato de época em um arquivo para várias entradas usando shell script

Converter data em formato de época em um arquivo para várias entradas usando shell script

abaixo está o formato em um arquivo. alguém pode ajudar na conversão das duas primeiras colunas que estão no formato de data para o formato de época?

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702058496,0,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702058496,2,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702058496,3,332,0,0,0,332,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702058496,4,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702062592,0,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702062592,2,0,0,0,0,919,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702062592,3,919,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702062592,4,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702066688,0,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702066688,2,0,0,0,0,0,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702066688,3,0,0,0,0,919,0,0

02-01-2018 05:44:00,2018-01-02 05:59:00,1,702066688,4,0,0,0,0,0,0,0

02-01-2018 10:44:00,2018-01-02 10:59:00,1,702058496,0,0,0,0,0,0,0,0

02-01-2018 10:44:00,2018-01-02 10:59:00,1,702058496,2,0,0,0,0,0,0,0

eu tentei com o seguinte comando.

aa= cat ATMIF.180102-0615 |grep -|awk -F ',' '{print $1}'; for i in cat ATMIF.180102-0615 |grep -|awk -F ',' '{print $1}';do date -d "$aa" "+%s"; feito

mas está dando erro como: date: invalid date 2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00' date: invalid date2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05 :44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018 -01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 10: 44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018- 01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44 :00\n2018-01-02 10:44:00\n2018-01-02 10:44:00' data: data inválida `2018-01-02 05:44:00\n2018-01-02 05:44: 00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01- 02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00\n2018-01-02 05:44:00 \n2018-01-02 05:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\ n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00\n2018-01-02 10:44:00'

Responder1

Isso seria suficiente, assumindo que os dados estão em:

IFS=,;while read line; do 
  ff=($line); 
  ff[0]=`date -d "${ff[0]}" +%s`;
  ff[1]=`date -d "${ff[1]}" +%s`;
  echo "${ff[*]}"; 
done < file.in > file.out

Nota: pode ser necessário adicionar 'UTC' no comando de data de acordo com o formato de origem.

Agora, para fazer isso no local, conforme solicitado pelo seu comentário, você precisa fazer um loop em todos os arquivos. Por exemplo, usando o comando find para corresponder todos os arquivos .csv recursivamente em /mypath/to-files/:

for file in $(find /mypath/to-files -name "*.csv"); do
  cp $file /tmp/tmp.csv
  IFS=,;while read line; do 
      ff=($line); 
      ff[0]=`date -d "${ff[0]}" +%s`;
      ff[1]=`date -d "${ff[1]}" +%s`;
      echo "${ff[*]}"; 
    done < /tmp/tmp.csv > $file
done

informação relacionada