![]() |
|
[Tut] [Collection] URL Decoding Methods - Printable Version +- Sick Gaming (https://sickgaming.net) +-- Forum: Programming (https://sickgaming.net/forum-76.html) +--- Forum: Python (https://sickgaming.net/forum-83.html) +--- Thread: [Tut] [Collection] URL Decoding Methods (/thread-96836.html) |
[Tut] [Collection] URL Decoding Methods - xSicKxBot - 08-22-2020 [Collection] URL Decoding Methods URL encoding “is a method to encode information in a Uniform Resource Identifier (URI)”. It is also called Percent-encoding because percentage symbols are used to encode certain reserved characters: Code: !Code: #Code: $Code: %Code: &Code: 'Code: (Code: )Code: *Code: +Code: ,Code: /Code: :Code: ;Code: =Code: ?Code: @Code: [Code: ]Code: %21Code: %23Code: %24Code: %25Code: %26Code: %27Code: %28Code: %29Code: %2ACode: %2BCode: %2CCode: %2FCode: %3ACode: %3BCode: %3DCode: %3FCode: %40Code: %5BCode: %5DThis article collects various ways to decode an URL encoded string. Let’s get started! Python 2 Code: $ alias urldecode='python -c "import sys, urllib as ul; \ print ul.unquote_plus(sys.argv[1])"' $ alias urlencode='python -c "import sys, urllib as ul; \ print ul.quote_plus(sys.argv[1])"'Source Here’s an example usage: Code: $ urldecode 'q+werty%3D%2F%3B'
q werty=/; $ urlencode 'q werty=/;'
q+werty%3D%2F%3BCode: $ alias urldecode='python3 -c "import sys, urllib.parse as ul; \ print(ul.unquote_plus(sys.argv[1]))"' $ alias urlencode='python3 -c "import sys, urllib.parse as ul; \ print (ul.quote_plus(sys.argv[1]))"'Here’s an example usage: Code: $ urldecode 'q+werty%3D%2F%3B'
q werty=/; $ urlencode 'q werty=/;'
q+werty%3D%2F%3BSource sed Code: $ sed 's@+@ @g;s@%@\\x@g' file | xargs -0 printf "%b"Source sed with echo -e Code: $ sed -e's/%\([0-9A-F][0-9A-F]\)/\\\\\x\1/g' file | xargs echo -eSource sed with alias For convenience, you may want to use an alias: Code: $ alias urldecode='sed "s@+@ @g;s@%@\\\\x@g" | xargs -0 printf "%b"'If you want to decode, you can now simply use: Code: $ echo "http%3A%2F%2Fwww" | urldecode
http://wwwSource Bash Code: input="http%3A%2F%2Fwww"
decoded=$(printf '%b' "${input//%/\\x}")Source To handle pluses ( Code: +Code: sedCode: decoded=$(input=${input//+/ }; printf "${input//%/\\x}")Code: urlencode() { # urlencode <string> local length="${#1}" for (( i = 0; i < length; i++ )); do local c="${1:i:1}" case $c in [a-zA-Z0-9.~_-]) printf "$c" ;; *) printf '%%%02X' "'$c" ;; esac done
} urldecode() { # urldecode <string> local url_encoded="${1//+/ }" printf '%b' "${url_encoded//%/\\x}"
}Sources: bash + xxd Code: urlencode() { local length="${#1}" for (( i = 0; i < length; i++ )); do local c="${1:i:1}" case $c in [a-zA-Z0-9.~_-]) printf "$c" ;; *) printf "$c" | xxd -p -c1 | while read x;do printf "%%%s" "$x";done esac
done
}Sources: PHP Code: $ echo oil+and+gas | php -r 'echo urldecode(fgets(STDIN));' // Or: php://stdin
oil and gasSource PHP Library Code: php -r 'echo urldecode("oil+and+gas");'Source Perl Code: decoded_url=$(perl -MURI::Escape -e 'print uri_unescape($ARGV[0])' "$encoded_url")Source Perl to Process File Code: perl -i -MURI::Escape -e 'print uri_unescape($ARGV[0])' fileSource awk Code: awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..Sources:
Python 2 urllib.unquote The Code: urllib.unquoteCode: urllibCode: decoded_url=$(python2 -c 'import sys, urllib; print urllib.unquote(sys.argv[1])' "$encoded_url")You can also use it to modify a file: Code: python2 -c 'import sys, urllib; print urllib.unquote(sys.stdin.read())' <file >file.new &&
mv -f file.new fileSource: https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding Python 3 urllib.parse.unquote If you run Python 3 on your system (like most people would), use the alternative function Code: urllib.parse.unquoteCode: decoded_url=$(python3 -c 'import sys, urllib.parse; print(urllib.parse.unquote(sys.argv[1]))' "$encoded_url")Again, you can use the function to process a file as follows: Code: python3 -c 'import sys, urllib; print(urllib.parse.unquote(sys.stdin.read()))' <file >file.new &&
mv -f file.new fileSource: https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding Perl URI::Escape The Code: URI::EscapeCode: decoded_url=$(perl -MURI::Escape -e 'print uri_unescape($ARGV[0])' "$encoded_url")You can use the function to process a file as follows: Code: perl -i -MURI::Escape -e 'print uri_unescape($ARGV[0])' fileSource: https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding Perl One-Liner Without Installing Modules Code: $ perl -pe 's/\%(\w\w)/chr hex $1/ge'Here’s a usage example: Code: $ echo '%21%22' | perl -pe 's/\%(\w\w)/chr hex $1/ge'
!"Source: https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding Bash Regex Code: $ function urldecode() { : "${*//+/ }"; echo -e "${_//%/\\x}"; }Now, you can use the function as a command like this: Code: $ urldecode https%3A%2F%2Fgoogle.com%2Fsearch%3Fq%3Durldecode%2Bbash
https://google.com/search?q=urldecode+bashIf you need to assign some variables, use this strategy: Code: $ x="http%3A%2F%2Fstackoverflow.com%2Fsearch%3Fq%3Durldecode%2Bbash"
$ y=$(urldecode "$x")
$ echo "$y"
http://stackoverflow.com/search?q=urldecode+bashSource: https://stackoverflow.com/questions/6250698/how-to-decode-url-encoded-string-in-shell GNU Awk Code: #!/usr/bin/awk -fn
@include "ord"
BEGIN { RS = "%.."
}
{ printf "%s", $0 if (RT != "") { printf "%s", chr("0x" substr(RT, 2)) }
}Source: https://stackoverflow.com/questions/6250698/how-to-decode-url-encoded-string-in-shell References
|