テキスト ファイルを検索して重複をすべて見つけ、重複セット内の 1 つをオリジナルとしてマークする必要があります。Windows のコマンドラインから実行することをお勧めします。
サンプル入力テキスト
AAA;BBB;CCC;DDD
001791dc;acbc38;27730a86e0e2;3000
0017cc6a;b7bc38;27730a86e276;1056
0017ce72;b8bc38;27730a86e28b;98544
0017d066;b8bc38;27730a86e295;5211
00188a4c;dcbc38;27730a871636;45896
00188c4a;ddbc38;27730a871640;12535
0019f6c0;1ebc38;27730a874d4e;3000
001c7652;7abc38;27730a87892a;1056
001c9312;7ebc38;27730a87899e;52117
001fbdda;f4bc38;27730a87f785;3000
希望する出力テキスト
AAA;BBB;CCC;DDD
0017cc6a;b7bc38;27730a86e276;1056;ORIGINAL
001c7652;7abc38;27730a87892a;1056;DUPLICATE
00188c4a;ddbc38;27730a871640;12535;ORIGINAL
001791dc;acbc38;27730a86e0e2;3000;ORIGINAL
0019f6c0;1ebc38;27730a874d4e;3000;DUPLICATE
001fbdda;f4bc38;27730a87f785;3000;DUPLICATE
00188a4c;dcbc38;27730a871636;45896;ORIGINAL
0017d066;b8bc38;27730a86e295;5211;ORIGINAL
001c9312;7ebc38;27730a87899e;52117;ORIGINAL
0017ce72;b8bc38;27730a86e28b;98544;ORIGINAL
これは私がこれまで試してきたもので、期待どおりの出力が得られませんでした。
@echo off
setlocal EnableDelayedExpansion EnableExtensions
REM == Drive where application is excuted ==
set dr=%~d0
set t1=temp_hedr_11.txt
set t2=temp_no_22.txt
set t3=temp_sort_33.txt
set t4=temp_sortd_hdr_44.txt
REM == Remove Header from Text file ==
For /F "tokens=1,2,3 delims=$ usebackq skip=1" %%A IN ( "new1.txt" ) do ( Echo %%A>> "!t2!" )
REM == Sort Text file ==
Sort "!t2!">> "!t3!"
REM == Add Header to Text file ==
Echo AAA;BBB;CCC;DDD>"!t4!"
Type "!t3!" >> "!t4!"
REM == Delete Temp Files ==
IF EXIST "!t2!" ( DEL "!t2!" )
IF EXIST "!t3!" ( DEL "!t3!" )
REM == Check for Duplicates==
set /a a_counter=1
REM == First line has header ==
For /F "tokens=1-20 delims=; usebackq skip=1" %%A IN ( "!t4!" ) do (
REM Removes Space at the end
Set "F589R=%%D"
set dup_chr11=!F589R: =!
Set "BB2B=%%A;%%B;%%C;!dup_chr11!"
set /a b_counter=1
Echo !BB2B!;ORIGINAL>> "MATCHING.txt"
For /F "tokens=1-20 delims=; usebackq" %%A IN ( "!t4!" ) do (
REM Removes Space at the end
set "HY4UJ8=%%D"
set dup_chr22=!HY4UJ8: =!
Set "B2VC=%%A;%%B;%%C;!dup_chr22!"
IF !a_counter! GTR !b_counter! ( IF "!dup_chr11!"=="!dup_chr22!" ( ECHO !B2VC!;DUPLICATE >> "MATCHING.txt" ))
set /a b_counter+=1
)
set /a a_counter+=1
)
Endlocal
答え1
何度も試行錯誤を繰り返して、私はこれを自分で機能させることができました。
@echo off
setlocal EnableDelayedExpansion EnableExtensions
REM == Copy Last Column to the Beginning and Remove Header from text ==
For /F "tokens=1-20* delims=; usebackq skip=1" %%A IN ( "new1.txt" ) do (
REM =Removes Space at the end=
Set "HF59R=%%D"
set kijd=!HF59R: =!
Echo !kijd!;%%A;%%B;%%C;!kijd!>> "33.txt"
)
REM == Set duplicate checker value ==
For /F "tokens=1-20* delims=; usebackq" %%A IN ( "33.txt" ) do (
REM =Removes Space at the end=
Set "F589R=%%A"
set duplicate_chck=!F589R: =!
REM == Find all lines matching the duplicate_chck ==
Findstr "!duplicate_chck!;" "33.txt" > "44.txt"
set /a a_counter=1
REM == Write to final output file, which identifies which are duplicate or original ==
For /F "tokens=1-20* delims=; usebackq" %%A IN ( "44.txt" ) do (
Set "F5R=%%E"
set D5DD=!F5R: =!
IF !a_counter!==1 ( Echo %%B;%%C;%%D;!D5DD!;ORIGINAL>>Final_Output.txt )
IF !a_counter! GTR 1 ( Echo %%B;%%C;%%D;!D5DD!;DUPLICATE>>Final_Output.txt )
set /a a_counter+=1
)
REM == Finds all lines that do not match the duplicate_chck ==
Findstr /v "!duplicate_chck!;" "33.txt" > "55.txt"
REM == Delete/Replace file that has matching text with the not matching text ==
IF Exist "33.txt" ( DEL "33.txt" )
IF NOT Exist "33.txt" ( REN "55.txt" "33.txt" )
)
REM == Clean/Delete files no longer needed ==
IF Exist "33.txt" ( DEL "33.txt" )
IF Exist "44.txt" ( DEL "44.txt" )
IF Exist "55.txt" ( DEL "55.txt" )
Endlocal