2GB 파일이 있습니다. 여기에는 헤더와 다수의 "이벤트" 구조가 있습니다. 시작 부분은 다음과 같습니다.
<run example>
<header>
5
This is header
</header>
<event = 22>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
4 1 4 3 1 0 1 0
0 0 0 0 0 1 1 8
0 1 0 2 1 5 2 0
2 1 3 7 3 1 1 0
1 0 10100 2 3 1 5 1
1 5 1 7 2 3 2 2
</evhead>
0 97
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 11 1 0 0 0 4.63012694434E+00 2.62561831936E+00 -2.31855757639E+01 2.37887130977E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 22 1 0 0 0 -4.63012694434E+00 -2.62561831936E+00 -4.31442423592E+00 3.71128690719E+00 -5.75956188088E+00
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 2 0 0 0 -2.16995636615E-14 -1.11022302463E-15 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 22 4 0 0 0 -4.60626572550E+00 -2.61208727495E+00 -2.23619853289E+00 5.74815342040E+00 0.00000000000E+00
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
</event>
전체 파일에는 이러한 "이벤트" 블록이 97000개 포함되어 있습니다. 그래서 나는 이 파일을 각각 헤더와 "이벤트" 블록의 10,000개를 포함하는 방식으로 10개의 파일로 분할하고 싶습니다. 모든 블록은 인덱스에 따라 다릅니다(무작위임). 물론 마지막 파일에는 7,000개의 블록만 포함됩니다.
다음과 같이 스택에서 여러 지침을 시도했습니다.https://stackoverflow.com/questions/8544197/splitting-a-file-in-linux-based-on-content https://stackoverflow.com/questions/8544197/splitting-a-file-in-linux-based-on-content 하지만 나에게는 효과가 없었습니다.
다음은 모든 테스트에 대한 파일의 좀 더 큰 예입니다(file_to_download):
<run example>
<header>
5
header
</header>
<event = 22>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
4 1 4 3 1 0 1 0
</evhead>
0 97
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
</event>
<event = 26>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
4 1 4 3 1 0 1 0
</evhead>
0 52
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
</event>
<event = 31>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
4 1 4 3 1 0 1 0
0 0 0 0 0 1 1 8
</evhead>
0 92
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 11 1 0 0 0 4.39003604933E+00 4.97037860337E+00 -2.04926313413E+01 2.15389187176E+01 5.10000000000E-04
</event>
<event = 37>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
</evhead>
0 77
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 11 1 0 0 0 7.91768942174E+00 3.75815788575E+00 -2.09569980000E+01 2.27158385693E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
</event>
<event = 41>
<evhead>
8
3 1 2 0 0 0 0 0
0 0 1 0 1 30 0 1
4 1 4 3 1 0 1 0
</evhead>
0 122
3 11 0 0 3 4 1.94791176123E-14 0.00000000000E+00 -2.75000000000E+01 2.75000000047E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 2212 0 0 5 0 -1.94791176123E-14 0.00000000000E+00 9.20000000000E+02 9.20000478451E+02 9.38270000000E-01
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
3 11 1 0 0 0 -3.63469912393E+00 3.95372353695E+00 -1.62133507727E+01 1.70796870892E+01 5.10000000000E-04
0.00000000000E+00 0.00000000000E+00 0.00000000000E+00 0.00000000000E+00
</event>
답변1
GNU를 사용하면 awk
:
BEGIN { fname = "/dev/null" }
/<header>/,/<\/header>/ { hdr = hdr $0 "\n"; next }
/^<event / {
events++
if(events % 10000 == 1) {
if(files++) close(fname)
fname = sprintf("file%02d.txt", files)
print hdr >fname
}
}
{ print >>fname }
실행하려면: 파일에 쓴 script.awk
후 다음을 실행하세요.
gawk -f script.awk file.txt