...
 
Commits (29)
    https://gitcode.net/opendilab/treevalue/-/commit/a40d97c0da5af6d65987ebee10402dec5e4042ae dev(hansbug): optimize graphviz visualization 2023-02-23T15:37:09+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/10ca5544e421a8043b765be97e845519f4a313ea dev(hansbug): update new size limit 2023-02-23T16:00:34+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/a5addeb274ea1d88933dabee9c48c030895feeb0 dev(narugo): refactor the renderings 2023-02-23T17:01:01+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/46e3c80e7d4886ca3ff1cd861b5fa791c41ab946 dev(hansbug): add jupyter repr support 2023-02-23T21:27:48+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/e45694219ed0d346729384638757fd81e141cd01 Merge branch 'main' into dev/repr 2023-02-24T14:45:01+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/eeb0c9b208816eae357ffd1aa097bc49eb252727 dev(narugo): add unittest for repr functions 2023-02-24T15:00:49+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/7c41463cd2e90008c32dfdc3ff1c873b0fd1a0ae dev(hansbug): add examples in form of jupyter notebook 2023-02-24T15:10:51+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/66f70a720fa1f31f301b0543812ba2f0fe1cdbec dev(hansbug): try debug length on windows 2023-02-24T15:12:42+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/af1c40c5e4eb28df3659331ebdc8a5e5b7c570d5 dev(hansbug): fix problem in png size on windows 2023-02-24T15:22:42+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/74b0038d9c8b625c9607a5be4f88d466061df859 dev(hansbug): add dot check when calling build graph 2023-02-24T15:42:10+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/f45a90495b79267370bccd9f14d4d946626da980 dev(narugo): add installation of graphviz in benchmark 2023-02-24T15:51:55+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/de4f6b8785c3d8473c8add8234b89f12d4034d47 dev(hansbug): add bold for non-value node 2023-02-24T18:02:46+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/82adadeb1c677a297dc1bc4d28780e87f29579b5 dev(hansbug): fix problem of unittest 2023-02-24T18:11:29+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/7cdcb26771dfd31101b247fce3a11503c62c7740 Merge branch 'main' into dev/repr 2023-02-27T00:03:49+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/df95a7a9cc8d7b85db6d0496c51e1ace7c7ed11a Merge branch 'main' into dev/repr 2023-03-05T15:28:30+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/80d4e9b59c14bfe0e8df157f755c6f4589f983ee fix(hansbug): fix bug of #82, add more unittests 2023-03-06T16:27:36+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/2de348633704f4df5427a39a08077ecae04edbe1 release(hansbug): use version 1.4.8 2023-03-06T17:15:58+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/947e13739675fc7367c963c2fab7903f147c74d9 Merge pull request #83 from opendilab/fix/treelize 2023-03-06T17:35:28+08:00 Hankson Bradley hansbug@buaa.edu.cn fix(hansbug): fix bug of #82, add more unittests https://gitcode.net/opendilab/treevalue/-/commit/2ded457dc2b06b4caf18f15347c67b1344dd5840 Merge branch 'main' into dev/repr 2023-03-06T18:14:26+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/576dea7385af5d0ad1d88613687236796f23be8c dev(hansbug): fix problem mentioned in #75 2023-03-07T00:06:31+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/2cc7d7a79ce8765c14ab8acd3723199356aa89c8 Merge pull request #75 from opendilab/dev/repr 2023-03-07T00:15:56+08:00 Hankson Bradley hansbug@buaa.edu.cn dev(hansbug): optimize graphviz visualization https://gitcode.net/opendilab/treevalue/-/commit/4891ef3cebe470e04b7fad2308208b1d35b835ec fix(hansbug): try fix build test, ci skip 2023-03-07T00:44:08+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/826b8a12b3c47c307b52bdbe3a1ad1d8b275fba2 dev(hansbug): fix this version 2023-03-07T01:01:08+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/0c7cbf0ac293a7b81e4c06bc9ad78f106463e23a dev(hansbug): update try release script 2023-03-07T01:05:14+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/9769033e7cd8318587a3a98e21f17611a9e0f6f5 dev(hansbug): fix bug in unittest 2023-03-07T01:11:54+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/271c36a43047f04dba27df5890e6fa990523d9e0 dev(hansbug): support common repr when dot not installed 2023-03-07T01:22:08+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/b930e657e7f9dd60c3cc99426bc9fe9d1f7632f1 dev(hansbug): do not install graphviz 2023-03-07T01:38:11+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/29c44a939199b80da714e9c4b571142b51e26353 dev(hansbug): update release script 2023-03-07T01:57:21+08:00 HansBug hansbug@buaa.edu.cn https://gitcode.net/opendilab/treevalue/-/commit/421d9363b607db852c2d5a9757463d2a8380be6c release(hansbug): use version 1.4.9 2023-03-07T02:17:00+08:00 HansBug hansbug@buaa.edu.cn
...@@ -66,17 +66,17 @@ jobs: ...@@ -66,17 +66,17 @@ jobs:
shell: bash shell: bash
run: | run: |
sudo apt-get update sudo apt-get update
sudo apt-get install -y tree cloc wget curl make sudo apt-get install -y tree cloc wget curl make graphviz
- name: Set up system dependences on Windows - name: Set up system dependences on Windows
if: ${{ env.OS_NAME == 'Windows' }} if: ${{ env.OS_NAME == 'Windows' }}
shell: bash shell: bash
run: | run: |
choco install tree cloc wget curl make zip choco install tree cloc wget curl make zip graphviz
- name: Set up system dependences on MacOS - name: Set up system dependences on MacOS
if: ${{ env.OS_NAME == 'MacOS' }} if: ${{ env.OS_NAME == 'MacOS' }}
shell: bash shell: bash
run: | run: |
brew install tree cloc wget curl make zip brew install tree cloc wget curl make zip graphviz
- name: Set up python ${{ matrix.python-version }} - name: Set up python ${{ matrix.python-version }}
uses: actions/setup-python@v4 uses: actions/setup-python@v4
with: with:
......
...@@ -5,7 +5,7 @@ on: ...@@ -5,7 +5,7 @@ on:
types: [ published ] types: [ published ]
jobs: jobs:
source_release: source_build:
name: Build and publish the source package name: Build and publish the source package
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
if: ${{ github.repository == 'opendilab/treevalue' }} if: ${{ github.repository == 'opendilab/treevalue' }}
...@@ -36,24 +36,17 @@ jobs: ...@@ -36,24 +36,17 @@ jobs:
- name: Build packages - name: Build packages
run: | run: |
make zip make zip
- name: Publish distribution 📦 to real PyPI ls -al dist
uses: pypa/gh-action-pypi-publish@master - name: Upload packed files to artifacts
with: uses: actions/upload-artifact@v2
password: ${{ secrets.PYPI_PASSWORD }}
verbose: true
skip_existing: true
- name: Upload distribution 📦 to github release
uses: svenstaro/upload-release-action@v2
with: with:
repo_token: ${{ secrets.GITHUB_TOKEN }} name: build-artifacts-all
file: dist/* path: ./dist/*
tag: ${{ github.ref }}
overwrite: false
file_glob: true
wheel_build: wheel_build:
name: Build the wheels name: Build the wheels
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
if: ${{ github.repository == 'opendilab/treevalue' }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
...@@ -112,86 +105,52 @@ jobs: ...@@ -112,86 +105,52 @@ jobs:
CIBW_SKIP: "pp* *musllinux*" CIBW_SKIP: "pp* *musllinux*"
CIBW_ARCHS: ${{ matrix.architecture }} CIBW_ARCHS: ${{ matrix.architecture }}
CIBW_PROJECT_REQUIRES_PYTHON: ~=${{ matrix.python }}.0 CIBW_PROJECT_REQUIRES_PYTHON: ~=${{ matrix.python }}.0
- name: Show the buildings - name: Show the buildings
shell: bash shell: bash
run: | run: |
ls -al ./wheelhouse ls -al ./wheelhouse
mv wheelhouse dist
- name: Upload packed files to artifacts - name: Upload packed files to artifacts
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v2
with: with:
name: build-artifacts-${{ matrix.os }}-cp${{ matrix.python }}-${{ matrix.architecture }} name: build-artifacts-all
path: ./wheelhouse path: ./dist/*
# the publishing can only be processed on linux system # the publishing can only be processed on linux system
wheel_publish: wheel_publish:
name: Publish the wheels to pypi name: Publish the wheels to pypi
runs-on: ubuntu-20.04 runs-on: ubuntu-20.04
needs: needs:
- source_build
- wheel_build - wheel_build
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os:
- 'ubuntu-20.04'
- 'windows-2019'
- 'macos-12'
python: python:
- '3.7'
- '3.8' - '3.8'
- '3.9'
- '3.10'
- '3.11'
architecture:
- x86_64
- arm64
- aarch64
- x86
- AMD64
exclude:
- os: ubuntu-20.04
architecture: arm64
- os: ubuntu-20.04
architecture: x86
- os: ubuntu-20.04
architecture: AMD64
- os: windows-2019
architecture: x86_64
- os: windows-2019
architecture: arm64
- os: windows-2019
architecture: aarch64
- os: macos-12
architecture: aarch64
- os: macos-12
architecture: x86
- os: macos-12
architecture: AMD64
- python: '3.7'
architecture: arm64
steps: steps:
- name: Download packed files to artifacts - name: Download packed files to artifacts
uses: actions/download-artifact@v3 uses: actions/download-artifact@v3
with: with:
name: build-artifacts-${{ matrix.os }}-cp${{ matrix.python }}-${{ matrix.architecture }} name: build-artifacts-all
path: ./wheelhouse path: ./dist
- name: Show the buildings - name: Show the buildings
shell: bash shell: bash
run: | run: |
ls -al ./wheelhouse ls -al ./dist
- name: Publish distribution 📦 to real PyPI - name: Publish distribution 📦 to real PyPI
uses: pypa/gh-action-pypi-publish@master uses: pypa/gh-action-pypi-publish@master
with: with:
password: ${{ secrets.PYPI_PASSWORD }} password: ${{ secrets.PYPI_PASSWORD }}
verbose: true verbose: true
skip_existing: true skip_existing: true
packages_dir: wheelhouse/ packages_dir: dist/
- name: Upload distribution 📦 to github release - name: Upload distribution 📦 to github release
uses: svenstaro/upload-release-action@v2 uses: svenstaro/upload-release-action@v2
with: with:
repo_token: ${{ secrets.GITHUB_TOKEN }} repo_token: ${{ secrets.GITHUB_TOKEN }}
file: wheelhouse/* file: dist/*
tag: ${{ github.ref }} tag: ${{ github.ref }}
overwrite: false overwrite: false
file_glob: true file_glob: true
...@@ -3,7 +3,7 @@ name: Release Test ...@@ -3,7 +3,7 @@ name: Release Test
on: workflow_dispatch on: workflow_dispatch
jobs: jobs:
source_release: source_build:
name: Try package the source name: Try package the source
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
...@@ -40,8 +40,13 @@ jobs: ...@@ -40,8 +40,13 @@ jobs:
with: with:
name: build-artifacts-source-pack name: build-artifacts-source-pack
path: ./dist/* path: ./dist/*
- name: Upload packed files to artifacts
uses: actions/upload-artifact@v2
with:
name: build-artifacts-all
path: ./dist/*
wheel_release: wheel_build:
name: Try build the wheels name: Try build the wheels
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
...@@ -102,13 +107,42 @@ jobs: ...@@ -102,13 +107,42 @@ jobs:
CIBW_SKIP: "pp* *musllinux*" CIBW_SKIP: "pp* *musllinux*"
CIBW_ARCHS: ${{ matrix.architecture }} CIBW_ARCHS: ${{ matrix.architecture }}
CIBW_PROJECT_REQUIRES_PYTHON: ~=${{ matrix.python }}.0 CIBW_PROJECT_REQUIRES_PYTHON: ~=${{ matrix.python }}.0
- name: Show the buildings - name: Show the buildings
shell: bash shell: bash
run: | run: |
ls -al ./wheelhouse ls -al ./wheelhouse
mv wheelhouse dist
- name: Upload packed files to artifacts - name: Upload packed files to artifacts
uses: actions/upload-artifact@v3 uses: actions/upload-artifact@v3
with: with:
name: build-artifacts-${{ runner.os }}-cp${{ matrix.python }}-${{ matrix.architecture }} name: build-artifacts-${{ runner.os }}-cp${{ matrix.python }}-${{ matrix.architecture }}
path: ./wheelhouse/* path: ./dist/*
- name: Upload packed files to artifacts
uses: actions/upload-artifact@v2
with:
name: build-artifacts-all
path: ./dist/*
# the publishing can only be processed on linux system
fake_publish:
name: Fake Publish
runs-on: ubuntu-20.04
needs:
- wheel_build
- source_build
strategy:
fail-fast: false
matrix:
python:
- '3.8'
steps:
- name: Download packed files to artifacts
uses: actions/download-artifact@v3
with:
name: build-artifacts-all
path: ./dist
- name: Show the buildings
shell: bash
run: |
ls -al ./dist
.PHONY: clean
JUPYTER ?= $(shell which jupyter)
NBCONVERT ?= ${JUPYTER} nbconvert
SOURCE ?= .
IPYNBS := $(shell find ${SOURCE} -name *.ipynb | grep -v .ipynb_checkpoints)
clean:
for nb in ${IPYNBS}; do \
if [ -f $$nb ]; then \
$(NBCONVERT) --clear-output --inplace $$nb; \
fi; \
done;
\ No newline at end of file
# Examples of TreeValue
## Examples
Here are some TreeValue examples that can be viewed on Colab.
* Visualization: [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opendilab/treevalue/blob/main/examples/visualization.ipynb)
## Makefile
```shell
make clean # clean all output in notebook files
```
**Please make sure the output of all notebook files are cleared with `make clean` before committing**.
{
"cells": [
{
"cell_type": "markdown",
"id": "edb6ed95",
"metadata": {},
"source": [
"# Visualization of TreeValue"
]
},
{
"cell_type": "markdown",
"id": "9fb09e8c",
"metadata": {},
"source": [
"Create a simple `FastTreeValue` object."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b982cde7",
"metadata": {},
"outputs": [],
"source": [
"!pip install treevalue"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dc7e3dd2",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import torch\n",
"from treevalue import FastTreeValue\n",
"\n",
"t = FastTreeValue({\n",
" 'a': 1,\n",
" 'b': np.random.randint(0, 20, (3, 4)),\n",
" 'x': {\n",
" 'c': 3,\n",
" 'd': torch.randn(2, 3),\n",
" },\n",
"})"
]
},
{
"cell_type": "markdown",
"id": "8b654200",
"metadata": {},
"source": [
"See it"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "30771752",
"metadata": {},
"outputs": [],
"source": [
"t"
]
},
{
"cell_type": "markdown",
"id": "b2c1bb61",
"metadata": {},
"source": [
"Do some operations on `t`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc97351a",
"metadata": {},
"outputs": [],
"source": [
"1 / (t + 0.5)"
]
},
{
"cell_type": "markdown",
"id": "b25f3db4",
"metadata": {},
"source": [
"Create another `FastTreeValue` which is sharing storage with `t`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af6345dc",
"metadata": {},
"outputs": [],
"source": [
"t2 = FastTreeValue({'ppp': t.x, 'x': {'t': t, 'y': t.x}})"
]
},
{
"cell_type": "markdown",
"id": "e039d8d9",
"metadata": {},
"source": [
"Just put `t` and `t2` into one diagram."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "57446c40",
"metadata": {},
"outputs": [],
"source": [
"from treevalue import graphics\n",
"\n",
"graphics((t, 't'), (t2, 't2'))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
...@@ -16,12 +16,15 @@ t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}) ...@@ -16,12 +16,15 @@ t1 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}})
t2 = FastTreeValue({'a': 1, 'b': {2, 4}, 'x': {'c': [1, 3], 'd': 4}}) t2 = FastTreeValue({'a': 1, 'b': {2, 4}, 'x': {'c': [1, 3], 'd': 4}})
t3 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': t2.b, 'd': t2.x.c}}) t3 = FastTreeValue({'a': 1, 'b': 2, 'x': {'c': t2.b, 'd': t2.x.c}})
g = graphics( if cmdv('dot'):
(t1, 't1'), (t2, 't2'), (t3, 't3'), g = graphics(
title='This is title for g.', (t1, 't1'), (t2, 't2'), (t3, 't3'),
cfg=dict(bgcolor='#ffffff00'), title='This is title for g.',
dup_value=(list,), cfg=dict(bgcolor='#ffffff00'),
) dup_value=(list,),
)
else:
g = None
@pytest.mark.unittest @pytest.mark.unittest
...@@ -43,6 +46,7 @@ class TestEntryCliGraph: ...@@ -43,6 +46,7 @@ class TestEntryCliGraph:
assert os.path.exists('test_graph.gv') assert os.path.exists('test_graph.gv')
assert os.path.getsize('test_graph.gv') <= 2500 assert os.path.getsize('test_graph.gv') <= 2500
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_simple_code_graph_to_stdout(self): def test_simple_code_graph_to_stdout(self):
runner = CliRunner() runner = CliRunner()
with runner.isolated_filesystem(): with runner.isolated_filesystem():
...@@ -198,6 +202,7 @@ class TestEntryCliGraph: ...@@ -198,6 +202,7 @@ class TestEntryCliGraph:
assert 'first title' not in content assert 'first title' not in content
assert 'This is title for g.' in content assert 'This is title for g.' in content
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_cfg(self): def test_cfg(self):
runner = CliRunner() runner = CliRunner()
with runner.isolated_filesystem(): with runner.isolated_filesystem():
......
...@@ -3,6 +3,7 @@ from operator import __mul__ ...@@ -3,6 +3,7 @@ from operator import __mul__
import pytest import pytest
from treevalue import FastTreeValue
from treevalue.tree import func_treelize, TreeValue, method_treelize, classmethod_treelize, delayed from treevalue.tree import func_treelize, TreeValue, method_treelize, classmethod_treelize, delayed
...@@ -401,3 +402,29 @@ class TestTreeFuncFunc: ...@@ -401,3 +402,29 @@ class TestTreeFuncFunc:
'v': {'a': 12, 'b': 25, 'x': {'c': 38, 'd': 51}}, 'v': {'a': 12, 'b': 25, 'x': {'c': 38, 'd': 51}},
}) })
assert cnt_1 == 4 assert cnt_1 == 4
def test_return_treevalue(self):
def func(x):
return FastTreeValue({
'x': x, 'y': x ** 2,
})
f = FastTreeValue({
'x': func,
'y': {
'z': func,
}
})
v = FastTreeValue({'x': 2, 'y': {'z': 34}})
assert f(v) == FastTreeValue({
'x': {
'x': v.x,
'y': v.x ** 2,
},
'y': {
'z': {
'x': v.y.z,
'y': v.y.z ** 2,
}
}
})
import unittest
from functools import reduce from functools import reduce
from operator import __mul__ from operator import __mul__
from typing import Type from typing import Type
import numpy as np import numpy as np
import pytest import pytest
from hbutils.testing import cmdv
from treevalue.tree import func_treelize, TreeValue, raw, mapping, delayed, FastTreeValue from treevalue.tree import func_treelize, TreeValue, raw, mapping, delayed, FastTreeValue
from ..tree.base import get_treevalue_test from ..tree.base import get_treevalue_test
...@@ -606,6 +608,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]): ...@@ -606,6 +608,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]):
assert t2.x.c is not t.x.c assert t2.x.c is not t.x.c
assert t2.x.d is not t.x.d assert t2.x.d is not t.x.d
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_graph(self): def test_graph(self):
t = treevalue_class({ t = treevalue_class({
'a': [4, 3, 2, 1], 'a': [4, 3, 2, 1],
...@@ -619,6 +622,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]): ...@@ -619,6 +622,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]):
graph = t.graph('t') graph = t.graph('t')
assert len(graph.source) <= 2290 assert len(graph.source) <= 2290
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_graphics(self): def test_graphics(self):
t = treevalue_class({ t = treevalue_class({
'a': [4, 3, 2, 1], 'a': [4, 3, 2, 1],
...@@ -646,7 +650,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]): ...@@ -646,7 +650,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]):
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_1.source) <= 4960 assert len(graph_1.source) <= 5000 # a value for svg in windows
graph_2 = treevalue_class.graphics( graph_2 = treevalue_class.graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
...@@ -655,7 +659,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]): ...@@ -655,7 +659,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]):
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_2.source) <= 5480 assert len(graph_2.source) <= 5600
graph_3 = treevalue_class.graphics( graph_3 = treevalue_class.graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
...@@ -673,7 +677,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]): ...@@ -673,7 +677,7 @@ def get_fasttreevalue_test(treevalue_class: Type[FastTreeValue]):
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_4.source) <= 3780 assert len(graph_4.source) <= 4000
graph_6 = treevalue_class.graphics( graph_6 = treevalue_class.graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
......
...@@ -23,7 +23,7 @@ class TestTreeTreeIntegration: ...@@ -23,7 +23,7 @@ class TestTreeTreeIntegration:
'a': np.random.randint(0, 10, (2, 3)), 'a': np.random.randint(0, 10, (2, 3)),
'b': { 'b': {
'x': np.asarray(233.0), 'x': np.asarray(233.0),
'y': np.random.randn(2, 3) 'y': np.random.randn(2, 3) + 1,
} }
}) })
r1 = double(t1) r1 = double(t1)
...@@ -40,7 +40,7 @@ class TestTreeTreeIntegration: ...@@ -40,7 +40,7 @@ class TestTreeTreeIntegration:
'a': np.random.randint(0, 10, (2, 3)), 'a': np.random.randint(0, 10, (2, 3)),
'b': { 'b': {
'x': np.asarray(233.0), 'x': np.asarray(233.0),
'y': np.random.randn(2, 3) 'y': np.random.randn(2, 3) + 1,
} }
}) })
r2 = double(t2) r2 = double(t2)
......
import pickle import pickle
import re import re
import unittest
from typing import Type from typing import Type
import pytest import pytest
from hbutils.testing import OS, cmdv
from test.tree.tree.test_constraint import GreaterThanConstraint from test.tree.tree.test_constraint import GreaterThanConstraint
from treevalue import raw, TreeValue, delayed, ValidationError from treevalue import raw, TreeValue, delayed, ValidationError
...@@ -749,4 +751,34 @@ def get_treevalue_test(treevalue_class: Type[TreeValue]): ...@@ -749,4 +751,34 @@ def get_treevalue_test(treevalue_class: Type[TreeValue]):
assert newt1 == t1 assert newt1 == t1
assert newt1.constraint == t1.constraint assert newt1.constraint == t1.constraint
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_repr_svg(self):
t1 = get_demo_constraint_tree()
assert hasattr(t1, '_repr_svg_')
_repr_svg_ = t1._repr_svg_()
assert isinstance(_repr_svg_, str)
assert 4500 <= len(_repr_svg_) <= 4900
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_repr_png(self):
t1 = get_demo_constraint_tree()
assert hasattr(t1, '_repr_png_')
_repr_png_ = t1._repr_png_()
assert isinstance(_repr_png_, bytes)
if OS.windows:
assert 12000 <= len(_repr_png_) <= 16050
else:
assert 16050 <= len(_repr_png_) <= 20500
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_repr_jpeg(self):
t1 = get_demo_constraint_tree()
assert hasattr(t1, '_repr_jpeg_')
_repr_jpeg_ = t1._repr_jpeg_()
assert isinstance(_repr_jpeg_, bytes)
assert 10500 <= len(_repr_jpeg_) <= 14500
return _TestClass return _TestClass
import unittest
import numpy as np import numpy as np
import pytest import pytest
from hbutils.testing import cmdv
from treevalue import FastTreeValue, graphics from treevalue import FastTreeValue, graphics
...@@ -10,6 +13,7 @@ class MyFastTreeValue(FastTreeValue): ...@@ -10,6 +13,7 @@ class MyFastTreeValue(FastTreeValue):
@pytest.mark.unittest @pytest.mark.unittest
class TestTreeTreeGraph: class TestTreeTreeGraph:
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_graphics(self): def test_graphics(self):
t = MyFastTreeValue({ t = MyFastTreeValue({
'a': [4, 3, 2, 1], 'a': [4, 3, 2, 1],
...@@ -37,7 +41,7 @@ class TestTreeTreeGraph: ...@@ -37,7 +41,7 @@ class TestTreeTreeGraph:
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_1.source) <= 4960 assert len(graph_1.source) <= 5000
graph_2 = graphics( graph_2 = graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
...@@ -46,7 +50,7 @@ class TestTreeTreeGraph: ...@@ -46,7 +50,7 @@ class TestTreeTreeGraph:
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_2.source) <= 5480 assert len(graph_2.source) <= 5600
graph_3 = graphics( graph_3 = graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
...@@ -64,7 +68,7 @@ class TestTreeTreeGraph: ...@@ -64,7 +68,7 @@ class TestTreeTreeGraph:
title="This is a demo of 2 trees with dup value.", title="This is a demo of 2 trees with dup value.",
cfg={'bgcolor': '#ffffffff'}, cfg={'bgcolor': '#ffffffff'},
) )
assert len(graph_4.source) <= 3780 assert len(graph_4.source) <= 4000
graph_6 = graphics( graph_6 = graphics(
(t, 't'), (t1, 't1'), (t, 't'), (t1, 't1'),
......
import unittest
from shutil import which
from unittest.mock import patch
import pytest import pytest
from hbutils.testing import cmdv
from treevalue.utils import build_graph from treevalue.utils import build_graph
@pytest.fixture()
def no_dot():
def _no_dot_which(x):
if x == 'dot':
return None
else:
return which(x)
with patch('shutil.which', _no_dot_which):
yield
@pytest.mark.unittest @pytest.mark.unittest
class TestUtilsTree: class TestUtilsTree:
@unittest.skipUnless(cmdv('dot'), 'Dot installed only')
def test_build_graph(self): def test_build_graph(self):
t = {'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}} t = {'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}
g = build_graph((t, 't'), graph_title="Demo of build_graph.") g = build_graph((t, 't'), graph_title="Demo of build_graph.")
...@@ -14,17 +32,20 @@ class TestUtilsTree: ...@@ -14,17 +32,20 @@ class TestUtilsTree:
g2 = build_graph(t, graph_title="Demo 2 of build_graph.") g2 = build_graph(t, graph_title="Demo 2 of build_graph.")
assert "Demo 2 of build_graph." in g2.source assert "Demo 2 of build_graph." in g2.source
assert "<root_0>" in g2.source assert "root_0" in g2.source
assert "<root_0>.x" in g2.source
assert len(g2.source) <= 580 assert len(g2.source) <= 580
g3 = build_graph((t,), graph_title="Demo 3 of build_graph.") g3 = build_graph((t,), graph_title="Demo 3 of build_graph.")
assert "Demo 3 of build_graph." in g3.source assert "Demo 3 of build_graph." in g3.source
assert "<root_0>" in g3.source assert "root_0" in g3.source
assert "<root_0>.x" in g3.source
assert len(g3.source) <= 580 assert len(g3.source) <= 580
g4 = build_graph((), graph_title="Demo 4 of build_graph.") g4 = build_graph((), graph_title="Demo 4 of build_graph.")
assert "Demo 4 of build_graph." in g4.source assert "Demo 4 of build_graph." in g4.source
assert "node" not in g4.source assert "node" not in g4.source
assert len(g4.source) <= 110 assert len(g4.source) <= 110
def test_build_graph_without_dot(self, no_dot):
t = {'a': 1, 'b': 2, 'x': {'c': 3, 'd': 4}}
with pytest.raises(EnvironmentError):
_ = build_graph((t, 't'), graph_title="Demo of build_graph.")
...@@ -7,7 +7,7 @@ Overview: ...@@ -7,7 +7,7 @@ Overview:
__TITLE__ = "treevalue" __TITLE__ = "treevalue"
#: Version of this project. #: Version of this project.
__VERSION__ = "1.4.7" __VERSION__ = "1.4.9"
#: Short description of the project, will be included in ``setup.py``. #: Short description of the project, will be included in ``setup.py``.
__DESCRIPTION__ = 'A flexible, generalized tree-based data structure.' __DESCRIPTION__ = 'A flexible, generalized tree-based data structure.'
......
...@@ -57,6 +57,7 @@ cdef object _c_func_treelize_run(object func, list args, dict kwargs, _e_tree_mo ...@@ -57,6 +57,7 @@ cdef object _c_func_treelize_run(object func, list args, dict kwargs, _e_tree_mo
cdef list _a_args cdef list _a_args
cdef dict _a_kwargs cdef dict _a_kwargs
cdef object _a_ret
if not has_tree: if not has_tree:
_a_args = [] _a_args = []
for v in args: for v in args:
...@@ -72,7 +73,11 @@ cdef object _c_func_treelize_run(object func, list args, dict kwargs, _e_tree_mo ...@@ -72,7 +73,11 @@ cdef object _c_func_treelize_run(object func, list args, dict kwargs, _e_tree_mo
else: else:
_a_kwargs[k] = missing_func() _a_kwargs[k] = missing_func()
return func(*_a_args, **_a_kwargs) _a_ret = func(*_a_args, **_a_kwargs)
if isinstance(_a_ret, TreeValue):
return _a_ret._detach()
else:
return _a_ret
cdef dict _d_res = {} cdef dict _d_res = {}
cdef str ak cdef str ak
......
from functools import lru_cache from functools import lru_cache
from typing import Type, Callable, Union, Optional, Tuple from typing import Type, Callable, Union, Optional, Tuple
from graphviz import Digraph, nohtml from graphviz import Digraph
from hbutils.reflection import post_process, dynamic_call, freduce from hbutils.reflection import post_process, dynamic_call, freduce
from .tree import TreeValue from .tree import TreeValue
...@@ -134,6 +134,13 @@ _GENERIC_N = 36 ...@@ -134,6 +134,13 @@ _GENERIC_N = 36
_GENERIC_S = _GENERIC_N // 3 _GENERIC_S = _GENERIC_N // 3
def _custom_repr(x):
if isinstance(x, (str,)):
return repr(x)
else:
return str(x)
def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None, def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None,
dup_value: Union[bool, Callable, type, Tuple[Type, ...]] = False, dup_value: Union[bool, Callable, type, Tuple[Type, ...]] = False,
repr_gen: Optional[Callable] = None, repr_gen: Optional[Callable] = None,
...@@ -182,9 +189,9 @@ def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None, ...@@ -182,9 +189,9 @@ def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None,
return build_graph( return build_graph(
*trees, *trees,
node_id_gen=_node_tag, node_id_gen=_node_tag,
graph_title=title or "<untitled>", graph_title=title,
graph_cfg=cfg or {}, graph_cfg=cfg or {},
repr_gen=repr_gen or (lambda x: nohtml(repr(x))), repr_gen=repr_gen or (lambda x: _custom_repr(x)),
iter_gen=lambda n: iter(n.items()) if isinstance(n, TreeValue) else None, iter_gen=lambda n: iter(n.items()) if isinstance(n, TreeValue) else None,
node_cfg_gen=_dict_call_merge(lambda n, p, np, pp, is_node, is_root, root: { node_cfg_gen=_dict_call_merge(lambda n, p, np, pp, is_node, is_root, root: {
'fillcolor': _shape_color(root[2]), 'fillcolor': _shape_color(root[2]),
...@@ -193,7 +200,7 @@ def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None, ...@@ -193,7 +200,7 @@ def graphics(*trees, title: Optional[str] = None, cfg: Optional[dict] = None,
'style': 'filled', 'style': 'filled',
'shape': 'diamond' if is_root else ('ellipse' if is_node else 'box'), 'shape': 'diamond' if is_root else ('ellipse' if is_node else 'box'),
'penwidth': 3 if is_root else 1.5, 'penwidth': 3 if is_root else 1.5,
'fontname': "Times-Roman bold" if is_node else "Times-Roman", 'fontname': "monospace bold" if is_node else "monospace",
}, (node_cfg_gen or (lambda: {}))), }, (node_cfg_gen or (lambda: {}))),
edge_cfg_gen=_dict_call_merge(lambda n, p, np, pp, is_node, root: { edge_cfg_gen=_dict_call_merge(lambda n, p, np, pp, is_node, root: {
'arrowhead': 'vee' if is_node else 'dot', 'arrowhead': 'vee' if is_node else 'dot',
......
...@@ -49,6 +49,8 @@ cdef class TreeValue: ...@@ -49,6 +49,8 @@ cdef class TreeValue:
cpdef void validate(self) except* cpdef void validate(self) except*
cdef object _get_tree_graph(self)
cdef str _prefix_fix(object text, object prefix) cdef str _prefix_fix(object text, object prefix)
cdef str _title_repr(TreeStorage st, object type_) cdef str _title_repr(TreeStorage st, object type_)
cdef object _build_tree(TreeStorage st, object type_, str prefix, dict id_pool, tuple path) cdef object _build_tree(TreeStorage st, object type_, str prefix, dict id_pool, tuple path)
......
...@@ -2,6 +2,7 @@ ...@@ -2,6 +2,7 @@
# cython:language_level=3 # cython:language_level=3
import os import os
import shutil
from collections.abc import Sized, Container, Reversible, Mapping from collections.abc import Sized, Container, Reversible, Mapping
from operator import itemgetter from operator import itemgetter
...@@ -70,6 +71,12 @@ cdef class ValidationError(Exception): ...@@ -70,6 +71,12 @@ cdef class ValidationError(Exception):
return f"Validation failed on {self._cons!r} at position {self._path!r}{os.linesep}" \ return f"Validation failed on {self._cons!r} at position {self._path!r}{os.linesep}" \
f"{type(self._error).__name__}: {self._error}" f"{type(self._error).__name__}: {self._error}"
cdef inline bool _check_dot_installed():
if shutil.which('dot'):
return True
else:
return False
cdef class TreeValue: cdef class TreeValue:
r""" r"""
Overview: Overview:
...@@ -972,6 +979,32 @@ cdef class TreeValue: ...@@ -972,6 +979,32 @@ cdef class TreeValue:
else: else:
return self._type(self._st, constraint=to_constraint([constraint, self.constraint])) return self._type(self._st, constraint=to_constraint([constraint, self.constraint]))
@cython.final
cdef inline object _get_tree_graph(self):
from .graph import graphics
return graphics((self, '<root>'))
@cython.binding(True)
def _repr_svg_(self):
if _check_dot_installed():
return self._get_tree_graph().pipe(format='svg', encoding='utf-8')
else:
return None
@cython.binding(True)
def _repr_png_(self):
if _check_dot_installed():
return self._get_tree_graph().pipe(format='png')
else:
return None
@cython.binding(True)
def _repr_jpeg_(self):
if _check_dot_installed():
return self._get_tree_graph().pipe(format='jpeg')
else:
return None
cdef str _prefix_fix(object text, object prefix): cdef str _prefix_fix(object text, object prefix):
cdef list lines = [] cdef list lines = []
cdef int i cdef int i
......
import html
import re import re
import shutil
from functools import wraps from functools import wraps
from queue import Queue from queue import Queue
from typing import Optional, Mapping, Any, Callable from typing import Optional, Mapping, Any, Callable
...@@ -6,8 +8,6 @@ from typing import Optional, Mapping, Any, Callable ...@@ -6,8 +8,6 @@ from typing import Optional, Mapping, Any, Callable
from graphviz import Digraph from graphviz import Digraph
from hbutils.reflection import dynamic_call, post_process from hbutils.reflection import dynamic_call, post_process
from .random import random_hex_with_timestamp
def _title_flatten(title): def _title_flatten(title):
title = re.sub(r'[^a-zA-Z0-9_]+', '_', str(title)) title = re.sub(r'[^a-zA-Z0-9_]+', '_', str(title))
...@@ -69,6 +69,35 @@ def _root_process(root, index): ...@@ -69,6 +69,35 @@ def _root_process(root, index):
return root, '<root_%d>' % (index,), index return root, '<root_%d>' % (index,), index
_LEFT_ALIGN_LINESEP = r'\l'
def _custom_html_escape(x):
s = html.escape(x) \
.replace(' ', '&ensp;') \
.replace('\r\n', _LEFT_ALIGN_LINESEP) \
.replace('\n', _LEFT_ALIGN_LINESEP) \
.replace('\r', _LEFT_ALIGN_LINESEP)
if len(x.splitlines()) > 1 and not s.endswith(_LEFT_ALIGN_LINESEP):
s += _LEFT_ALIGN_LINESEP
return s
_DOT_NOT_INSTALL_TEXT = """
Command 'dot' not found in this environment.
Please install graphviz first, with:
- Ubuntu: apt install -y graphviz
- Windows: choco install graphviz
- MacOS: brew install graphviz
More detailed information can be found in documentation of graphviz python package: https://graphviz.readthedocs.io/en/stable/#installation
""".lstrip()
def _check_dot_installed():
if not shutil.which('dot'):
raise EnvironmentError(_DOT_NOT_INSTALL_TEXT)
def build_graph(*roots, node_id_gen: Optional[Callable] = None, def build_graph(*roots, node_id_gen: Optional[Callable] = None,
graph_title: Optional[str] = None, graph_name: Optional[str] = None, graph_title: Optional[str] = None, graph_name: Optional[str] = None,
graph_cfg: Optional[Mapping[str, Any]] = None, graph_cfg: Optional[Mapping[str, Any]] = None,
...@@ -100,11 +129,13 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None, ...@@ -100,11 +129,13 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None,
Returns: Returns:
- dot (:obj:`Digraph`): Graphviz directed graph object. - dot (:obj:`Digraph`): Graphviz directed graph object.
""" """
_check_dot_installed()
roots = [_root_process(root, index) for index, root in enumerate(roots)] roots = [_root_process(root, index) for index, root in enumerate(roots)]
roots = [item for item in roots if item is not None] roots = [item for item in roots if item is not None]
node_id_gen = dynamic_call(suffixed_node_id(node_id_gen or _default_node_id)) node_id_gen = dynamic_call(suffixed_node_id(node_id_gen or _default_node_id))
graph_title = graph_title or ('untitled_' + random_hex_with_timestamp()) graph_title = graph_title or ''
graph_name = graph_name or _title_flatten(graph_title) graph_name = graph_name or _title_flatten(graph_title)
graph_cfg = _no_none_value(graph_cfg or {}) graph_cfg = _no_none_value(graph_cfg or {})
...@@ -125,7 +156,7 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None, ...@@ -125,7 +156,7 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None,
root_node_id = node_id_gen(root, None, [], [], True) root_node_id = node_id_gen(root, None, [], [], True)
if root_node_id not in _queued_node_ids: if root_node_id not in _queued_node_ids:
graph.node( graph.node(
name=root_node_id, label=root_title, name=root_node_id, label=_custom_html_escape(root_title),
**node_cfg_gen(root, None, [], [], True, True, root_info) **node_cfg_gen(root, None, [], [], True, True, root_info)
) )
_queue.put((root_node_id, root, (root, root_title, root_index), [])) _queue.put((root_node_id, root, (root, root_title, root_index), []))
...@@ -145,14 +176,14 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None, ...@@ -145,14 +176,14 @@ def build_graph(*roots, node_id_gen: Optional[Callable] = None,
_current_label = repr_gen(_current_node, _current_path) _current_label = repr_gen(_current_node, _current_path)
if _current_id not in _queued_node_ids: if _current_id not in _queued_node_ids:
graph.node(_current_id, label=_current_label, graph.node(_current_id, label=_custom_html_escape(_current_label),
**node_cfg_gen(_current_node, _parent_node, _current_path, _parent_path, **node_cfg_gen(_current_node, _parent_node, _current_path, _parent_path,
_is_node, False, _root_info)) _is_node, False, _root_info))
if iter_gen(_current_node, _current_path): if iter_gen(_current_node, _current_path):
_queue.put((_current_id, _current_node, _root_info, _current_path)) _queue.put((_current_id, _current_node, _root_info, _current_path))
_queued_node_ids.add(_current_id) _queued_node_ids.add(_current_id)
if (_parent_id, _current_id, key) not in _queued_edges: if (_parent_id, _current_id, key) not in _queued_edges:
graph.edge(_parent_id, _current_id, label=key, graph.edge(_parent_id, _current_id, label=_custom_html_escape(key),
**edge_cfg_gen(_current_node, _parent_node, _current_path, _parent_path, **edge_cfg_gen(_current_node, _parent_node, _current_path, _parent_path,
_is_node, _root_info)) _is_node, _root_info))
_queued_edges.add((_parent_id, _current_id, key)) _queued_edges.add((_parent_id, _current_id, key))
......