wassname/xbsjsonedit
1
0
Fork 0
mirror of https://github.com/wassname/xbsjsonedit.git synced 2026-07-14 11:20:05 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'v1.1'

Browse Source
This commit is contained in:
cjnaz
2020-06-08 13:10:32 -07:00
parent bbbb3c6a53 9ad190705e
commit 535ff6e4fb
2 changed files with 248 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+96 -19
View File
@@ -1,32 +1,68 @@
# xbsjsonedit
xbsjsonedit is a limited tool for modifying backup data dumped from the
xbsjsonedit is a tool for working with backup data dumped from the
[xBrowserSync App](https://www.xbrowsersync.org/) ([GitHub](https://github.com/xBrowserSync)).
It was originally developed for cleanup up bookmark replication problems in early releases
of the App. This tool may be useful for pruning duplicate bookmarks and general maintenance.
If you want it to do other functions, grab the code and go for it.
xbsjsonedit supports both interactive and command line modes/features. Interactive mode is targeted at
editing your bookmark content, specifically looking for duplicate bookmarks and errors introduced by earlier versions
of xBrowserSync.
Command line (CLI) modes support searching and printing select views of your bookmark data. The `--html` mode outputs a
browser-loadable web page with your bookmarks organized by tag name. See more info below.
## Notable changes in the latest release
V1.10 200608
- Added `--html` command line mode which generates html output of your tagged bookmarks.
- Changed the command line interface so that the input file defaults to the newest xbs_backup*.json file in your specified
directory (per the JSON_PATH_DEFAULT assignment in the code). The input file may also be specified using the
`--jsonbackup` command line switch.
- Added `SearchTerm` command line mode feature, which prints matches within tags, and with the `--bookmark` switch prints matches
within bookmark titles and URLs.
## Usage
```
$ ./xbsjsonedit -h
usage: xbsjsonedit [-h] [--print] [--tags TAGS] [--names] [-V] Infile
usage: xbsjsonedit [-h] [--bookmarks] [--print] [--tags TAGS] [--names]
[--html HTML] [--jsonbackup JSONBACKUP] [-V]
[SearchTerm]
xBrowserSync json backup editor
Usages:
Command line modes:
xbsjsonedit news prints bookmarks tagged with 'news'
xbsjsonedit news --bookmarks prints bookmarks tagged with 'news' and any bookmarks or URLs containing 'news'
xbsjsonedit --print prints all bookmarks with their hierarchy
xbsjsonedit --tags 5 prints only tag names that are used at least 5 times
xbsjsonedit --tags 5 --names prints tag names and their bookmarks that are used at least 5 times
xbsjsonedit --tags 0 --names prints tag names and their bookmark that are used exactly 1 time
xbsjsonedit --html <outfile.html> writes tagged bookmarks to the specified browsable file
Interactive mode:
xbsjsonedit Enters interactive mode
The path to the xbs_backup*.json file defaults to the newest in the download directory. --jsonbackup overrides the default.
positional arguments:
Infile json backup file.
SearchTerm Print bookmarks tagged with this text (CLI mode).
optional arguments:
-h, --help show this help message and exit
--print, -p Print bookmark hierarchy (redirect to less or a file).
--tags TAGS, -t TAGS Print tags list filtered by tags with a minimum <n> number of uses. n=0 prints only single use tags.
--names, -n Used with --tags to enable printing the titles/names for each tag.
--bookmarks, -b Print bookmarks containing or tagged with the SearchTerm (CLI mode).
--print, -p Print complete bookmark hierarchy (redirect to less or a file) (CLI mode).
--tags TAGS, -t TAGS Print tags list filtered by tags with a minimum <n> number of uses. n=0 prints only single use tags. (CLI mode)
--names, -n Used with --tags to enable printing the titles/names for each tag. (CLI mode)
--html HTML Path of html output file.
--jsonbackup JSONBACKUP, -j JSONBACKUP
Path to json backup file. Default is newest version at </mnt/share/Downloads/xbs_backup*.json>.
-V, --version Return version number and exit.
```
## Internal command menu
## Usage notes for Interactive mode
```
$ ./xbsjsonedit xBrowserSyncBackup_201811232150.json
$ ./xbsjsonedit
---------------------------------------------------------------------
Options:
@@ -53,21 +89,18 @@ Options:
w: Write out the bookmarks data to a file
q: Quit/exit (Do a Write first!)
Enter option:
```
## Usage notes
Five modes are supported Listing/Deleting based on:
By default, the most recent `xbs_backup*.json` file in your `JSON_PATH_DEFAULT` directory is opened if `--jsonbackup` is not specified on the command line. Customize the `JSON_PATH_DEFAULT` Windows and/or Linux settings in the code.
The `p` Print function prints the full list of bookmarks in folder hierarchical form, listing the ID number and Title of each folder and bookmark.
Interactive mode supports five modes of Listing/Deleting based on:
- A search string in bookmarks or folders
- Duplicate URLs in bookmarks - such as the same URL existing in two or more folders
- Duplicate folder names
- Folders that have tags - there is no value for folders having tags
- Empty folders - which may arise if all bookmarks were deleted from a folder
The Print function prints the full list of bookmarks in folder hierarchical form, listing the ID number and Title of each folder and bookmark. The `--Print` switch may be used for getting a good visual dump of the bookmarks in less or an editor for reference while using the interactive mode. The tags on each bookmark are listed as well.
The `--tags <n>` switch prints each tag where the number of bookmarks with that tag >= `<n>`. Optionally include the `--names` switch to list the bookmark titles/names for each tag. `--tags 0` is a special case that prints only tags that have exactly one occurrence, which may be useful for finding extraneous/remnant tags usage.
A lower case letter `t/g/d/f/x/y` will list the offenders, while an upper case letter
`T/G/D/F/X/Y` will allow for selective deletes of the offenders. The lower case List operation need not be run before running the upper case Delete operation.
Delete functions support individual bookmark selection, or all that match the selection mode.
@@ -80,6 +113,49 @@ The output file defaults to the Infile + "_OUT", and can be specified during the
- **Note** [_may no longer be the case_]: xBrowserSync will allow you to restore to (blast) your current syncID, so you may want to Disable Sync, then create a new sync before restoring the modified bookmarks.
- **NOTE** that tags are not restored by xBrowserSync if sync is disabled. To recover, simply enable sync and do another restore.
## Usage nodes for Command Line Interface (CLI) mode
By default, the most recent `xbs_backup*.json` file in your `JSON_PATH_DEFAULT` directory is opened if `--jsonbackup` is not specified on the command line. Customize the `JSON_PATH_DEFAULT` Windows and/or Linux settings in the code.
The `--Print` switch may be used for getting a good visual dump of the bookmarks in less or an editor for reference while using the interactive mode. (Same output as 'p' print in interactive mode.)
If the optional SearchTerm command line argument is specified, all _tags_ containing the SearchTerm substring are printed, along with the names/titles of the bookmarks that have the matching tags. If the `--bookmarks` switch is also specified, then the names/titles of bookmarks that have the substring in their name, URL, or tag are also printed. For example, SearchTerm `cam` matches tags `camping` and `webcam`, and several bookmarks:
```
$ ./xbsjsonedit cam --bookmarks
***** Bookmarks tagged with <cam>:
[camping] (2):
All Stays camping locations
Big Bend National Park Texas
[webcam] (5):
NAU web cam 1
ADEQ Webcams
AZ Game and Fish Webcams
ADOT traffic cams
Flagstaff webcams
***** Bookmarks containing or tagged with <cam>:
Cambria Bicycle Outfitter - We Outfit the world in bicycles and bicycle parts
Camping World - Home Page
NAU web cam 1
Fry's Home Electronics | Computer Parts & Accessories, Software, Games, TVs, Cameras - Frys.com
Newegg.com - Computer Parts, Laptops, Electronics, HDTVs, Digital Cameras and More!
ADEQ Webcams
AZ Game and Fish Webcams
ADOT traffic cams
Flagstaff webcams
All Stays camping locations
Big Bend National Park Texas
```
The `--tags <n>` switch prints each tag where the number of bookmarks with that tag >= `<n>`. Optionally include the `--names` switch to list the bookmark titles/names for each tag. `--tags 0` is a special case that prints only tags that have exactly one occurrence, which may be useful for finding extraneous/remnant tags usage.
The `--html <outfile.html>` switch outputs a web page to the specified file that contains all tags with two (default) or more bookmarks. `--tags <n>` may be specified to override the default minimum tag usage count.
- Note that untagged bookmarks, and bookmarks with tags that are used less than the `--tags` count (default 2), are not output to the outfile.html.
- <outfile.html> is statically generated on-demand from a backup file, and thus does track bookmark changes made within xBrowserSync.
- You probably do not want to post this output file on a publicly visible server.
- The `--html` mode/feature is intended as a prototype web page that xBrowserSync may output natively in a future release, so it may not be feature-complete for your needs.
## Known issues:
@@ -88,6 +164,7 @@ The output file defaults to the Infile + "_OUT", and can be specified during the
## Version history
- 200608 v1.1 Added cli search and html features, defaulted .json file to be the newest in the download directory.
- 200330 v1.0 Merged pull request #3 from mblais (add unbuffered getch input; add 'all' option), Reworked --tags and --names switches.
- 191206 v0.3 Added tags dump
- 191205 v0.2 Updated to xbrowsersync v1.5 and Python 3.x ONLY
xbsjsonedit
+152 -36
View File
@@ -1,12 +1,29 @@
#!/usr/bin/env python3
"""xBrowserSync json backup editor"""
"""xBrowserSync json backup editor
__version__ = "v1.0 200330"
Usages:
Command line modes:
xbsjsonedit news prints bookmarks tagged with 'news'
xbsjsonedit news --bookmarks prints bookmarks tagged with 'news' and any bookmarks or URLs containing 'news'
xbsjsonedit --print prints all bookmarks with their hierarchy
xbsjsonedit --tags 5 prints only tag names that are used at least 5 times
xbsjsonedit --tags 5 --names prints tag names and their bookmarks that are used at least 5 times
xbsjsonedit --tags 0 --names prints tag names and their bookmark that are used exactly 1 time
xbsjsonedit --html <outfile.html> writes tagged bookmarks to the specified browsable file
Interactive mode:
xbsjsonedit Enters interactive mode
The path to the xbs_backup*.json file defaults to the newest in the download directory. --jsonbackup overrides the default.
"""
__version__ = "v1.1 200608"
#==========================================================
#
# Chris Nelson 2018 - 2019
# Chris Nelson 2018 - 2020
#
# 200608 v1.1 Added cli search and html features, defaulted .json file to be the newest in the download directory.
# 200330 v1.0 Merged pull request #3 from mblais (add unbuffered getch input; add 'all' option), Reworked --tags and --names switches.
# 191206 v0.3 Added tags dump
# 191205 v0.2 Updated to xbrowsersync v1.5 and Python 3.x ONLY
@@ -18,11 +35,12 @@ __version__ = "v1.0 200330"
#==========================================================
import argparse
import os.path
import os
import json
import codecs
import sys
import io
import glob
# Configs / Constants
@@ -30,6 +48,16 @@ INDENT = 2
OFILESUFFIX = "_OUT"
NONE_SEARCH = "__NONE__"
if sys.platform == "win32":
JSON_PATH_DEFAULT = "Z:\\Downloads\\"
if "linux" in sys.platform: # <linux2> on Py2, <linux> on Py3
JSON_PATH_DEFAULT = "/mnt/share/Downloads/"
py_version = float(sys.version_info.major) + float(sys.version_info.minor)/10
if py_version < 3.6:
print ("Not supported on Python versions before 3.6.")
sys.exit()
# Global items
url_dict = {}
folder_dict = {}
@@ -45,14 +73,55 @@ def main():
global folder_dict
global tags_dict
with io.open(args.Infile, encoding='utf8', errors="replace") as json_data:
with io.open(infile, encoding='utf8', errors="replace") as json_data:
json_dict = json.load(json_data)
term = NONE_SEARCH
# CLI mode
if args.print:
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term="", operation="printtree")
exit()
sys.exit()
if args.html is not None:
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term="", operation="GatherTags")
tags_count = 2
if args.tags > 0:
tags_count = args.tags
html_fout = io.open (args.html, 'wt')
html_fout.write("""<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>xbs Tags</title>
</head>
<body>
<a name="top"></a>
<h1>Tagged bookmarks list</h1>
""")
html_fout.write(f'(Tags used {tags_count} or more times)<br>\n')
for tag in sorted(tags_dict.keys()):
if len(tags_dict[tag]) >= tags_count:
html_fout.write(f' <b><a href="#{tag}">{tag}</a></b><br>\n')
html_fout.write(" <br>\n")
for tag in sorted(tags_dict.keys()):
if len(tags_dict[tag]) >= tags_count:
html_fout.write(f' <a name="{tag}"></a><H3>{tag} ({len(tags_dict[tag])}):</H3>\n')
html_fout.write(' <BLOCKQUOTE>\n')
for bookmark in tags_dict[tag]:
html_fout.write (' <A HREF="{}" target="_blank">{}</A><BR>\n'.format(bookmark['url'], bookmark['title']))
html_fout.write(' <a href="#top">(top)</a>\n')
html_fout.write(' </BLOCKQUOTE>\n')
html_fout.write(" </body> </html>")
html_fout.close()
sys.exit()
if args.tags > -1:
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term="", operation="GatherTags")
for tag in sorted(tags_dict.keys()):
@@ -69,8 +138,33 @@ def main():
for bookmark in tags_dict[tag]:
print (" {:<4} - {}".format(bookmark["id"], bookmark["title"]))
print()
exit()
sys.exit()
if args.SearchTerm is not None:
_search_term = args.SearchTerm.lower()
print (f"***** Bookmarks tagged with <{_search_term}>:")
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term="", operation="GatherTags")
for tag in sorted(tags_dict.keys()):
if _search_term in tag.lower():
print (f" [{tag}] ({len(tags_dict[tag])}):")
for bookmark in tags_dict[tag]:
print (f" {bookmark['title']}")
print()
if args.bookmarks:
match_cnt = 0
print (f"\n***** Bookmarks containing or tagged with <{_search_term}>:")
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term=_search_term, operation="search2")
sys.exit()
if args.bookmarks: # and not args.SearchTerm
print ("A SearchTerm is required with use of the --bookmark switch.")
sys.exit()
# Interactive mode
while (1):
match_cnt = 0
change_cnt = 0
@@ -103,7 +197,7 @@ Options:
q: Quit/exit (Do a Write first!)
""".format(term))
select = prompt("Enter option: ", valid="stTgGdDfFxXyYwq")
select = prompt("Enter option: ", valid="pstTgGdDfFxXyYwq")
if select == 'p':
digin(parent=json_dict["xbrowsersync"]["data"]["bookmarks"], parent_id="", path="/", search_term="", operation="printtree")
@@ -166,17 +260,17 @@ Options:
print ("\nMatches: {} Deletes: {}\n".format(match_cnt, change_cnt))
elif select == 'w':
ans = prompt_long("Output file name (default <{}>: ".format(args.Infile + OFILESUFFIX))
ans = prompt_long("Output file name (default <{}>: ".format(infile + OFILESUFFIX))
if ans == "":
ans = args.Infile + OFILESUFFIX
ans = infile + OFILESUFFIX
with io.open(ans, "w", encoding='utf8') as ofile:
json.dump(json_dict, ofile, ensure_ascii=False, indent=2)
elif select == 'q':
if prompt("Quit now? (Remember to write any changes first!) ", valid="yn") == 'y':
exit()
sys.exit()
else:
print ("Shouldn't have gotten here!")
exit()
sys.exit()
def digin(parent, parent_id, path, search_term, operation, commit=False, indent=""):
@@ -198,7 +292,6 @@ def digin(parent, parent_id, path, search_term, operation, commit=False, indent=
print ("{:<4} > {}{}".format(item["id"], indent, item["title"]))
if operation == "SearchFolders":
# local_changes = False
if search_term in item["title"].lower():
print ("{:<4} > {}{}".format(item["id"], indent, item["title"]))
match_cnt += 1
@@ -249,7 +342,7 @@ def digin(parent, parent_id, path, search_term, operation, commit=False, indent=
if len(item["children"]) == 0:
print ("\n{:4} - {} >>> {}".format(
item["id"],
path, #[3:],
path,
item["title"]))
match_cnt += 1
if commit:
@@ -295,7 +388,7 @@ def digin(parent, parent_id, path, search_term, operation, commit=False, indent=
if operation == "DupURLs":
log_urls (item["url"], path, item["id"], parent, parent_id, item_index)
if operation == "search":
if operation == "search" or operation == "search2":
match = False
if search_term in item["url"].lower() or search_term in item["title"].lower():
match = True
@@ -307,16 +400,19 @@ def digin(parent, parent_id, path, search_term, operation, commit=False, indent=
if match:
match_cnt += 1
print ("\n{:4} - {} >>> {}\n url: <{}>".format(
item["id"],
path[3:],
item["title"],
item["url"]))
if "tags" in item:
if item["tags"] == None:
print (" tags = None !!!")
else:
print (" tags: <{}>".format(item["tags"]))
if operation == "search":
print ("\n{:4} - {} >>> {}\n url: <{}>".format(
item["id"],
path[3:],
item["title"],
item["url"]))
if "tags" in item:
if item["tags"] == None:
print (" tags = None !!!")
else:
print (" tags: <{}>".format(item["tags"]))
if operation == "search2":
print (f" {item['title']}")
if commit:
if do_all:
@@ -339,9 +435,9 @@ def digin(parent, parent_id, path, search_term, operation, commit=False, indent=
if item["tags"] != None:
for tag in item["tags"]:
if tag not in tags_dict:
tags_dict[tag] = [{"title":item["title"], "id":item["id"]}]
tags_dict[tag] = [{"title":item["title"], "id":item["id"], "url":item["url"]}]
else:
tags_dict[tag].append({"title":item["title"], "id":item["id"]})
tags_dict[tag].append({"title":item["title"], "id":item["id"], "url":item["url"]})
@@ -349,7 +445,7 @@ path_dict = {}
def collect_items (path, parent, index):
"""Collect items for later deletion by delete_items."""
global path_dict
# print (path) # "/ > [xbs] Toolbar > Chris' > Blogs166"
# print (path) # "/ > [xbs] Toolbar > Chris' > Blogs166"
# print (parent) # List of dictionaries of bookmarks [{}, {}, {}]
# print (index) # Index within the list - 2
if path not in path_dict:
@@ -565,22 +661,42 @@ getch = _Getch()
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument('Infile',
help="json backup file.")
parser.add_argument('SearchTerm', nargs='?', # optional positional argument
help="Print bookmarks tagged with this text (CLI mode).")
parser.add_argument('--bookmarks', '-b', action='store_true',
help="Print bookmarks containing or tagged with the SearchTerm (CLI mode).")
parser.add_argument('--print', '-p', action='store_true',
help="Print bookmark hierarchy (redirect to less or a file).")
help="Print complete bookmark hierarchy (redirect to less or a file) (CLI mode).")
parser.add_argument('--tags', '-t', type=int, default=-1,
help="Print tags list filtered by tags with a minimum <n> number of uses. n=0 prints only single use tags.")
help="Print tags list filtered by tags with a minimum <n> number of uses. n=0 prints only single use tags. (CLI mode)")
parser.add_argument('--names', '-n', action='store_true',
help="Used with --tags to enable printing the titles/names for each tag.")
help="Used with --tags to enable printing the titles/names for each tag. (CLI mode)")
parser.add_argument('--html', type=str,
help=f"Path of html output file.")
parser.add_argument('--jsonbackup', '-j', type=str,
help=f"Path to json backup file. Default is newest version at <{JSON_PATH_DEFAULT}xbs_backup*.json>.")
parser.add_argument('-V', '--version',
help="Return version number and exit.",
action='version',
version='%(prog)s ' + __version__)
args = parser.parse_args()
if not os.path.exists(args.Infile):
print ("Can't find the input file <{}>".format(args.Infile))
exit()
infile = args.jsonbackup
if infile is None:
infile = "__none__"
list_of_files = glob.glob(f'{JSON_PATH_DEFAULT}xbs_backup*.json')
# if is_Linux:
# list_of_files = glob.glob(f'{JSON_PATH_DEFAULT_LINUX}xbs_backup*.json')
# if is_Windows:
# list_of_files = glob.glob(f'{JSON_PATH_DEFAULT_WINDOWS}xbs_backup*.json')
# for file in list_of_files:
# print (file)
infile = max(list_of_files, key=os.path.getctime)
# print ("latest: ", infile)
# exit()
if not os.path.exists(infile):
print ("Can't find the input file <{}>".format(infile))
sys.exit()
main()