Propose of new structure and associated scripts #1
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| 1) Won't be good idea to ask for e.g. <sample>.description.txt and | ||
| <sample>.specification.txt for each sample? | ||
| => done | ||
|
|
||
| 2) Won't be good idea to allow skip a sample from automatic testing | ||
| (because it is for GUI demonstration)? | ||
| => <sample>.no_pdml => done | ||
| => <sample>.no_txt => done | ||
|
|
||
| 3) There should be easy way how to describe which wireshark version | ||
| processed stored PDML file. | ||
| Probably different outputs should be stored for main branches (2.0, 2.2, | ||
| ...). It is obvious that each version of wireshark will change (improve) | ||
| some outputs. Therefore older release will not be able to process it | ||
| correctly (PDML diff will fail). | ||
| I think that without output version you can't run regression tests | ||
| automatically. | ||
| => <FILE>_<VERSION>.pdml should be created => done | ||
| => <FILE>_<VERSION>.text should be created => done | ||
| => output is validated to version of used tshark or older version => done | ||
|
|
||
| 4) There should be easy way how to run custom wireshark version without | ||
| modifying Makefile. I have multiple versions for testing. I think | ||
| environment variable solve it. | ||
| => done | ||
|
|
||
| 5) There should be easy way how to describe which filter should be used | ||
| for preprocessing PDML. Now only filter.xsl is used. | ||
| => filter.xsl in directory with sample is used => done | ||
|
|
||
| 6) Rules must be written. | ||
|
|
||
| 7) There should be a procedure which checks whether repository/new sample | ||
| contains expected information. Something like git review procedure for | ||
| wireshark source/gerrit. | ||
| => make verify_repository | ||
|
|
||
| 8) Run -T text and -T pdml | ||
| => done | ||
|
|
||
| 9) It should be possible to set parameters for tshark run (e.g. decode as). | ||
| => optional file <file>.args => done | ||
|
|
||
| --------------------------- | ||
|
|
||
| There are two levels of directory structure in tests/. It looks like <PROTO>/<sample_description>. First level is name of protocol which is tested. Second level is name/description of sample. | ||
| Protocol name shall be name of procotol which is going to be tested. The easiest way is to name is same way as display filter in wireshark names it - including upper layer protocols (e.g. rtp or rtp.ed137a). | ||
| Name of test depends on author, but should somehow describe the tested subject. When expected description (directory) is already there, add number to it (e.g. dns, dns-1, dns-2). | ||
|
|
||
| Sample file should be stored as .pcap.gz or .pcapng.gz. | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| #!/bin/bash | ||
|
|
||
| TSHARK_EXECUTABLE="$1" | ||
| SAMPLE_DIR="$2" | ||
| TYPE="$3" | ||
| REQ_VERSION="$4" | ||
|
|
||
| ${TSHARK_EXECUTABLE} --version > /dev/null 2> /dev/null | ||
| if [ "$?" != "0" ]; then | ||
| echo "Executable for tshark doesn't exists (${TSHARK_EXECUTABLE})" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Creating output '${TYPE}' for ${SAMPLE_DIR}:" | ||
| FILE=`basename "${SAMPLE_DIR}"` | ||
|
|
||
| TSHARK_VERSION=`${TSHARK_EXECUTABLE} --version | head -1 | cut -d' ' -f 3 | cut -d'.' -f1,2` | ||
| if [ -n "${REQ_VERSION}" ]; then | ||
| if [ "${REQ_VERSION}" != "${TSHARK_VERSION}" ]; then | ||
| echo " FAILED, required tshark version do not match running version" | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| cd "${SAMPLE_DIR}" | ||
|
|
||
| if [ -f "${FILE}.pcap.gz" ]; then | ||
| FILE_PCAP="${FILE}.pcap.gz" | ||
| elif [ -f "${FILE}.pcapng.gz" ]; then | ||
| FILE_PCAP="${FILE}.pcapng.gz" | ||
|
There was a problem hiding this comment. There are capture formats other than (compressed) pcap like android logcat, etc. What do you think about using a single extension (like FOO.pcap or FOO.cap) even if it is compressed? Otherwise we might have a lot of files here. Alternatively, we can stick to the original convention of looking for FOO given FOO.pdml (e.g. dns.pcapng.pdml) |
||
| else | ||
| echo " No sample for ${SAMPLE_DIR}" | ||
| exit 0 | ||
| fi | ||
|
|
||
| TSHARK_ARGS= | ||
| if [ -r "${FILE}.args" ]; then | ||
| TSHARK_ARGS=`cat "${FILE}.args"` | ||
| fi | ||
|
|
||
| OUTPUT_FILE="${FILE}_${TSHARK_VERSION}.${TYPE}" | ||
|
|
||
| XTYPE=${TYPE} | ||
| XARGS= | ||
| if [ "${TYPE}" == "pdml1" ]; then | ||
| XTYPE=pdml | ||
| XARGS= | ||
| elif [ "${TYPE}" == "pdml2" ]; then | ||
| XTYPE=pdml | ||
| XARGS=-2 | ||
| fi | ||
|
|
||
| if [ ! -f "${OUTPUT_FILE}" -o ${FILE_PCAP} -nt ${OUTPUT_FILE} ]; then | ||
| "${TSHARK_EXECUTABLE}" $TSHARK_ARGS -T ${XTYPE} ${XARGS} -r "${FILE_PCAP}" > "${OUTPUT_FILE}".tmp | ||
|
There was a problem hiding this comment. Missed opportunity for parallelism here I think, now single-pass cannot run together with -2 |
||
| if [ "$?" -eq "0" ]; then | ||
| if [ "${XTYPE}" == "pdml" ]; then | ||
| mv -f "${OUTPUT_FILE}.tmp" "${OUTPUT_FILE}.tmp2" | ||
| xsltproc filter.xsl "${OUTPUT_FILE}.tmp2" > "${OUTPUT_FILE}.tmp" | ||
| if [ "$?" -ne "0" ]; then | ||
| rm -f "${OUTPUT_FILE}.tmp" | ||
| rm -f "${OUTPUT_FILE}.tmp2" | ||
| echo " FAILED, file ${SAMPLE_DIR}/${OUTPUT_FILE}" | ||
| exit 1 | ||
| fi | ||
| rm -f "${OUTPUT_FILE}.tmp2" | ||
| fi | ||
| mv "${OUTPUT_FILE}.tmp" "${OUTPUT_FILE}" | ||
| echo " OK, file ${SAMPLE_DIR}/${OUTPUT_FILE}" | ||
| exit 0 | ||
| else | ||
| rm -f "${OUTPUT_FILE}.tmp" | ||
| echo " FAILED, file ${SAMPLE_DIR}/${OUTPUT_FILE}" | ||
| exit 1 | ||
| fi | ||
| else | ||
| echo " SKIPPED, already exists and is up to date (${SAMPLE_DIR}/${OUTPUT_FILE})" | ||
| exit 0 | ||
| fi | ||
|
|
||
| #* | ||
| #* Editor modelines - http://www.wireshark.org/tools/modelines.html | ||
| #* | ||
| #* Local variables: | ||
| #* c-basic-offset: 4 | ||
| #* tab-width: 4 | ||
| #* indent-tabs-mode: nil | ||
| #* End: | ||
| #* | ||
| #* vi: set shiftwidth=4 tabstop=4 expandtab: | ||
| #* :indentSize=4:tabSize=4:noTabs=true: | ||
| #* | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,113 @@ | ||
| #!/bin/bash | ||
|
|
||
| TSHARK_EXECUTABLE="$1" | ||
| FILE="$2" | ||
| TYPE="$3" | ||
| shift | ||
| shift | ||
| shift | ||
|
|
||
| ${TSHARK_EXECUTABLE} --version > /dev/null 2> /dev/null | ||
| if [ "$?" != "0" ]; then | ||
| echo "Executable for tshark doesn't exists (${TSHARK_EXECUTABLE})" | ||
| exit 0 | ||
| fi | ||
|
|
||
| DIR=`dirname "${FILE}"` | ||
|
|
||
| echo -n "Processing ${FILE}.${TYPE}: " | ||
|
|
||
| TSHARK_VERSION=`${TSHARK_EXECUTABLE} --version | head -1 | cut -d' ' -f 3 | cut -d'.' -f1,2` | ||
| if [ -n "${REQ_VERSION}" ]; then | ||
| if [ "${REQ_VERSION}" != "${TSHARK_VERSION}" ]; then | ||
| echo " FAILED, required tshark version do not match running version" | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| if [ -f "${FILE}.pcap.gz" ]; then | ||
| FILE_PCAP="${FILE}.pcap.gz" | ||
| elif [ -f "${FILE}.pcapng.gz" ]; then | ||
| FILE_PCAP="${FILE}.pcapng.gz" | ||
| else | ||
| echo " No sample for ${FILE}" | ||
| exit 0 | ||
| fi | ||
|
|
||
| TSHARK_ARGS= | ||
| if [ -r "${FILE}.args" ]; then | ||
| TSHARK_ARGS=`cat "${FILE}.args"` | ||
| fi | ||
|
|
||
| OUTPUT_FILE="${FILE}.${TYPE}.current" | ||
|
|
||
| LAST_VER= | ||
| for x in $@; do | ||
| if [ -f "${FILE}_${x}.${TYPE}" ]; then | ||
| LAST_VER=$x | ||
| fi | ||
| if [ "${x}" == "${TSHARK_VERSION}" ]; then | ||
| break | ||
| fi | ||
| done | ||
|
|
||
| BASE_FILE="${FILE}_${LAST_VER}.${TYPE}" | ||
|
|
||
| if [ ! -f "${BASE_FILE}" ]; then | ||
| echo " No stored output up to version ${TSHARK_VERSION}." | ||
| fi | ||
|
|
||
| XTYPE=${TYPE} | ||
| XARGS= | ||
| if [ "${TYPE}" == "pdml1" ]; then | ||
| XTYPE=pdml | ||
| XARGS= | ||
| elif [ "${TYPE}" == "pdml2" ]; then | ||
| XTYPE=pdml | ||
| XARGS=-2 | ||
| fi | ||
| "${TSHARK_EXECUTABLE}" $TSHARK_ARGS -T ${XTYPE} ${XARGS} -r "${FILE_PCAP}" 1> "${OUTPUT_FILE}".tmp2 2>&1 | ||
| if [ "$?" -eq "0" ]; then | ||
| if [ "${XTYPE}" == "pdml" ]; then | ||
| xsltproc "${DIR}"/filter.xsl "${OUTPUT_FILE}.tmp2" > "${OUTPUT_FILE}.tmp" | ||
| if [ "$?" -ne "0" ]; then | ||
| echo " FAILED (${LAST_VER}/${TSHARK_VERSION})" | ||
| exit 1 | ||
| fi | ||
|
|
||
| diff "${BASE_FILE}" "${OUTPUT_FILE}.tmp" | ||
| if [ "$?" -ne "0" ]; then | ||
| echo " FAILED (${LAST_VER}/${TSHARK_VERSION})" | ||
| exit 1 | ||
| fi | ||
| else | ||
| mv "${OUTPUT_FILE}.tmp2" "${OUTPUT_FILE}.tmp" | ||
| fi | ||
|
|
||
| diff "${BASE_FILE}" "${OUTPUT_FILE}.tmp" | ||
| if [ "$?" -ne "0" ]; then | ||
| echo " FAILED (${LAST_VER}/${TSHARK_VERSION})" | ||
| exit 1 | ||
| fi | ||
|
|
||
| rm -f "${OUTPUT_FILE}.tmp2" | ||
| mv "${OUTPUT_FILE}.tmp" "${OUTPUT_FILE}" | ||
| echo " OK (${LAST_VER}/${TSHARK_VERSION})" | ||
| exit 0 | ||
| else | ||
| echo " FAILED (${LAST_VER}/${TSHARK_VERSION})" | ||
| exit 1 | ||
| fi | ||
|
|
||
| #* | ||
| #* Editor modelines - http://www.wireshark.org/tools/modelines.html | ||
| #* | ||
| #* Local variables: | ||
| #* c-basic-offset: 4 | ||
| #* tab-width: 4 | ||
| #* indent-tabs-mode: nil | ||
| #* End: | ||
| #* | ||
| #* vi: set shiftwidth=4 tabstop=4 expandtab: | ||
| #* :indentSize=4:tabSize=4:noTabs=true: | ||
| #* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| #!/bin/bash | ||
|
|
||
| SAMPLE_DIR="$1" | ||
| shift | ||
| CHECKED_VERSIONS=$@ | ||
|
|
||
| if [ $# -gt 1 ]; then | ||
| ONE_VERSION=0 | ||
| else | ||
| ONE_VERSION=1 | ||
| fi | ||
|
|
||
| CHECK_ERROR=0 | ||
|
|
||
| echo -n "Checking ${SAMPLE_DIR}: " | ||
| FILE=`basename "${SAMPLE_DIR}"` | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".pcap.gz" -a ! -f ""${SAMPLE_DIR}"/"${FILE}".pcapng.gz" ]; then | ||
| echo -e "\n PCAP sample is missing in ${SAMPLE_DIR} (${SAMPLE_DIR}/${FILE}.pcap.gz or ${SAMPLE_DIR}/${FILE}.pcapng.gz)" | ||
| CHECK_ERROR=1 | ||
| fi | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".description.txt" ]; then | ||
| echo -e "\n PCAP description is missing in ${SAMPLE_DIR} (${SAMPLE_DIR}/${FILE}.description.txt)" | ||
| CHECK_ERROR=1 | ||
| fi | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".requirements.txt" ]; then | ||
| echo -e "\n PCAP requirements are missing in ${SAMPLE_DIR} (${SAMPLE_DIR}/${FILE}.requirements.txt)" | ||
| CHECK_ERROR=1 | ||
| fi | ||
|
|
||
| # Check for TXT | ||
| FOUND=0 | ||
| for v in ${CHECKED_VERSIONS}; do | ||
| if [ -f ""${SAMPLE_DIR}"/${FILE}_${v}.text" ]; then | ||
| FOUND=1 | ||
| fi | ||
| done | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".no_txt" -a $FOUND == 0 ]; then | ||
| if [ "${ONE_VERSION}" == "1" ]; then | ||
| echo -e "\n TXT output ${SAMPLE_DIR}/${FILE}_${CHECKED_VERSIONS}.text is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| else | ||
| echo -e "\n TXT output ${SAMPLE_DIR}/${FILE}_<VERSION>.text for any version of ${CHECKED_VERSIONS} is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| fi | ||
| fi | ||
|
|
||
| # Check for PDML1 | ||
| FOUND=0 | ||
| for v in ${CHECKED_VERSIONS}; do | ||
| if [ -f ""${SAMPLE_DIR}"/${FILE}_${v}.pdml1" ]; then | ||
| FOUND=1 | ||
| fi | ||
| done | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".no_pdml1" -a $FOUND == 0 ]; then | ||
| if [ "${ONE_VERSION}" == "1" ]; then | ||
| echo -e "\n PDML1 output ${SAMPLE_DIR}/${FILE}_${CHECKED_VERSIONS}.pdml1 is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| else | ||
| echo -e "\n PDML1 output ${SAMPLE_DIR}/${FILE}_<VERSION>.pdml1 for any version of ${CHECKED_VERSIONS} is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| fi | ||
| fi | ||
|
|
||
| # Check for PDML2 | ||
| FOUND=0 | ||
| for v in ${CHECKED_VERSIONS}; do | ||
| if [ -f ""${SAMPLE_DIR}"/${FILE}_${v}.pdml2" ]; then | ||
| FOUND=1 | ||
| fi | ||
| done | ||
| if [ ! -f ""${SAMPLE_DIR}"/"${FILE}".no_pdml2" -a $FOUND == 0 ]; then | ||
| if [ "${ONE_VERSION}" == "1" ]; then | ||
| echo -e "\n PDML2 output ${SAMPLE_DIR}/${FILE}_${CHECKED_VERSIONS}.pdml2 is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| else | ||
| echo -e "\n PDML2 output ${SAMPLE_DIR}/${FILE}_<VERSION>.pdml2 for any version of ${CHECKED_VERSIONS} is missing in ${SAMPLE_DIR}" | ||
| CHECK_ERROR=1 | ||
| fi | ||
| fi | ||
|
|
||
| if [ "${CHECK_ERROR}" == "1" ]; then | ||
| echo -e " Check failed" | ||
| else | ||
| echo -e " Check OK" | ||
| fi | ||
| exit ${CHECK_ERROR} | ||
|
|
||
| #* | ||
| #* Editor modelines - http://www.wireshark.org/tools/modelines.html | ||
| #* | ||
| #* Local variables: | ||
| #* c-basic-offset: 4 | ||
| #* tab-width: 4 | ||
| #* indent-tabs-mode: nil | ||
| #* End: | ||
| #* | ||
| #* vi: set shiftwidth=4 tabstop=4 expandtab: | ||
| #* :indentSize=4:tabSize=4:noTabs=true: | ||
| #* |
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Simple standard DNS query |
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Decode as simple DNS request |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| <?xml version="1.0"?> | ||
| <pdml> | ||
| <packet> | ||
| <proto name="dns" showname="Domain Name System (query)" size="32" pos="150"> | ||
| <field name="dns.id" showname="Transaction ID: 0x4301" size="2" pos="150" show="0x00004301" value="4301"/> | ||
| <field name="dns.flags" showname="Flags: 0x0100 Standard query" size="2" pos="152" show="0x00000100" value="0100"> | ||
| <field name="dns.flags.response" showname="0... .... .... .... = Response: Message is a query" size="2" pos="152" show="0" value="0" unmaskedvalue="0100"/> | ||
| <field name="dns.flags.opcode" showname=".000 0... .... .... = Opcode: Standard query (0)" size="2" pos="152" show="0" value="0" unmaskedvalue="0100"/> | ||
| <field name="dns.flags.truncated" showname=".... ..0. .... .... = Truncated: Message is not truncated" size="2" pos="152" show="0" value="0" unmaskedvalue="0100"/> | ||
| <field name="dns.flags.recdesired" showname=".... ...1 .... .... = Recursion desired: Do query recursively" size="2" pos="152" show="1" value="FFFFFFFF" unmaskedvalue="0100"/> | ||
| <field name="dns.flags.z" showname=".... .... .0.. .... = Z: reserved (0)" size="2" pos="152" show="0" value="0" unmaskedvalue="0100"/> | ||
| <field name="dns.flags.checkdisable" showname=".... .... ...0 .... = Non-authenticated data: Unacceptable" size="2" pos="152" show="0" value="0" unmaskedvalue="0100"/> | ||
| </field> | ||
| <field name="dns.count.queries" showname="Questions: 1" size="2" pos="154" show="1" value="0001"/> | ||
| <field name="dns.count.answers" showname="Answer RRs: 0" size="2" pos="156" show="0" value="0000"/> | ||
| <field name="dns.count.auth_rr" showname="Authority RRs: 0" size="2" pos="158" show="0" value="0000"/> | ||
| <field name="dns.count.add_rr" showname="Additional RRs: 0" size="2" pos="160" show="0" value="0000"/> | ||
| <field name="" show="Queries" size="20" pos="162" value="02696e016d057961686f6f03636f6d0000010001"> | ||
| <field name="" show="in.m.yahoo.com: type A, class IN" size="20" pos="162" value="02696e016d057961686f6f03636f6d0000010001"> | ||
| <field name="dns.qry.name" showname="Name: in.m.yahoo.com" size="16" pos="162" show="in.m.yahoo.com" value="02696e016d057961686f6f03636f6d00"/> | ||
| <field name="dns.qry.name.len" showname="Name Length: 14" size="16" pos="162" show="14" value="02696e016d057961686f6f03636f6d00"/> | ||
| <field name="dns.count.labels" showname="Label Count: 4" size="16" pos="162" show="4" value="02696e016d057961686f6f03636f6d00"/> | ||
| <field name="dns.qry.type" showname="Type: A (Host Address) (1)" size="2" pos="178" show="1" value="0001"/> | ||
| <field name="dns.qry.class" showname="Class: IN (0x0001)" size="2" pos="180" show="0x00000001" value="0001"/> | ||
| </field> | ||
| </field> | ||
| </proto> | ||
| </packet> | ||
| </pdml> |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be quoted like
"${TSHARK_EXECUTABLE}"(same below)?