Skip to content

Python

Syntax#

Python has no end if or code blocks, for the visual appearance, We can use #endif to make clear that the if ends.

iterators#

iterate once#

A filter is a special iterable object you can iterate over. However, much like a generator, you can iterate over it only once. So, by calling list(filtered), you are iterating over each element of the filter object to generate the list. At this point, you've reached the end of the iterable and nothing more to return. e.g.

1
2
3
4
5
6
7
8
9
fibonacci = [0,1,1,2,3,5,8,13,21,34,55]
filt_odd = (filter(lambda x: x % 2, fibonacci))
f2=filt_odd
list_before = list(filt_odd)
print(list(f2)) # []

list_after = list(filt_odd)
print(list_before) # [1, 1, 3, 5, 13, 21, 55]
print(list_after) # []

performance#

Use any()+comprehension#

Instead of any()+lambda

The reason is that you've passed a generator expression to the any() function. Python needs to convert your generator expression to a generator function and that's why it performs slower. Because a generator function needs to call the __next__() method each time for generating the item and passing it to the any. This is while in a manual defined function you are passing the whole list to your function which has all the items prepared already. Also another bottleneck in your code which has more cost than extra calls on next is the way you do the comparison. As mentioned in comment the better equivalent of your manual function is:

source see also

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
# review: some of these is incorrect as it is copying a old list from pointer.
import time
# print(any(a))
# a=[True, False]*1000**3 # 1.83582e-05 / 17.403 seconds
# a=[True, False]*100**3 # 1.88351e-05 / 0.044 seconds
# a=[True]+[False]*1000**3  #2.38419e-05/ 13.936 seconds

# any + lambda ==  print(any(list(filter(lambda e: e==2,a))))
# a=[2]+[3]*1000**3  #60.69581/ 74.453 seconds
# a=[2]+[3]*1000**2  #60.69581/ 74.453 seconds
# a=[2]+[3]*100**3  #0.05992/ 0.099 seconds
# a=[2]+[3]*100**4  #6.03189/ 7.425 seconds
a=[2]+[3]*100**4  #2.5749/ 1.668 seconds

t = time.time()

print(any(True for e in a if e > 0))
print(f"elapsed: {time.time() - t} seconds")

syntax#

filter map

1
2
3
4
5
6
7
8
9
# Add two lists using map and lambda
numbers1 = [1, 2, 3]
numbers2 = [4, 5, 6]

result = map(lambda x, y: x + y, numbers1, numbers2)
print(list(result))

Output:
[5, 7, 9]

reduce

operators#

Walrus Operator#

「命名表达式」，别名「海象运算符」 因为 := 很像海象「眼睛小，长着两枚长长的牙」这个特点 有点可爱

list/array#

traverse#

d = {'name1' : 'pythontab', 'name2' : '.', 'name3' : 'com'} for key in d: print (key, ' value : ', d[key])

functions#

function must be claimed before access. return is not compulsory.

lambda function#

tag:匿名函数 lambda 函数的语法只包含一个语句，如下： lambda arg1,arg2,.....argn:expression（主要是看下面的例子）

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
# 普通 python 函数
def func(a,b,c):
return a+b+c
print func(1,2,3)
# 返回值为 6

# lambda 匿名函数 , Like arrow functions in JS
f = lambda a,b,c:a+b+c
print f(1,2,3)
# 返回结果为 6

random function#

1
2
3
4
5
6
7
8
import random
if __name__ == '__main__':
    a = list(range(10))
    random.shuffle(a)
    print(a)
    print('sorting...')
    b=qsort(a)
    print(b)

custom classes#

__new__ and __instance__#

cannot inherit from int#

Because all classes are derived from object and object is an abstract class, you cannot inherit from int. in __init__(self), if set self=1. in other functions, print(self) is still not 1.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
class MyClass(int):
    def __init__(self):
        print("__init__")
        self = 123
        print(self)
        self.f()

    def f(self):
        print("f")
        print(self)

x = MyClass()
x.f()

code style#

we are obligated (not obliged) to use 4 spaces indent (by black, autopep8). merge loops use array.

logic control#

1
2
3
4
5
# for i in range(0, len(nums)):
for i,x in enumerate(nums):

# if foo==1:
if foo:

debugging#

python unresolved import vscode

1
2
3
4
# I use pylance now. I added this. Problem solved.
"python.analysis.extraPaths": ["./path-to-code/"],
# I tried this before, but not working.
"python.autoComplete.extraPaths": ["./path-to-your-code"],

Python typing#

  • @2021-12-15
  • Python Typing - Type Hints & Annotations explains how python types, although it is for Python3.8.
  • Not easy part is generic type and callable type.
  • There's no "non-null assertion" in python, but inline if statement is a way to do it. details

Python 3.10 new features#

Python version control#

  • @2021-12-16
  • It seems current best solution is to use pyenv. It's like nvm for node.
  • @2021-12-20
  • uninstall old versions

Install Python(Mac)#

  • @2021-12-16
  • It's better to install with brew.

Class diagram in Python#

  • @2021-12-16
  • GodBa said

    I never draw UML diagram after leaving school :joy: . Usually python developers don't draw it that's probably the reason you can't find a good one

  • Most links in What's the best way to generate a UML diagram from Python source code? is not applicable.
  • Tried Pyreverse of Pylint: Graphs are only per-file. Didn't try project/package level.
  • Tried Epydoc: Can't run with py3.
  • Tried PyUML. Last release in 2009. It's for Eclipse only.
  • [ ] Try these
    • [ ] gaphor
    • [ ] Umbrello (an IDE)
    • [ ] PyNSource : Reverse engineer python source code into UML. Generated UML class diagrams can be displayed as ASCII Art or in a more typical graphical form. After a few years in a zombie state, we have just seen a new and improved version this 2019.
    • [ ] Epydoc : Generate UML documentation from python code (last release on 2008)
    • [ ] Lumpy . Python module that generates UML diagrams (currently object and class diagrams) from a running Python program. It is similar to a graphical debugger in the sense that it generates a visualization of the state of a running program, but it is different from a debugger in the sense that it tries to generate high-level visualizations that are compliant (at least in spirit) with standard UML
    • [ ] And others like ObjectDomain (commercial tool with forward and reverse engineering support for Python) and argoUML-python do not even seem to exist anymore.
    • [ ] GraphModels (django-command-extensions) creates a UML-like GraphViz dot file for the specified app name. Latest release from 2009.
    • [ ] pywebuml uses graphviz to create a UML class diagram representing your python (and also Java and C#) code. Latest release from 2012.
    • [ ] uml-to-django automatically generates the Django model and admin interface from a UML class diagram. My own UMLtoDjango service had a very similar tool, focusing on the automatic generation of CRUD interfaces for Django from a simple class diagram.

Online Python Env#

venv(Virtual Env)#

  • @2021-12-18
  • my py is aliased to python3 (some python version)
  • create a venv py -m venv [FOLDER_NAME|PATH]
  • activate a venv source [FOLDER_NAME|PATH]/bin/activate
  • update python version of venv py -m venv --upgrade [ENV_DIR]

Python module method override#

Turn Python module into Python class#

Python type assertion#

Import from other path#

  • @2022-01-02
  • Use . for current folder (Suggest to use on importing all local, non-pip package) and .. for parent folder, and ... for parent parent folder.
  • See origin

Relative import in Python3#

  • @2022-01-03
  • When there's relative import, need to run it as module (not as script) with python3 -m package.module
  • Relative imports in Python 3

Path of Project root#

CLI with Python#

  • @2022-01-04
  • Final solution: Typer
  • Tried: Click; pyCLI: deprecated in 2016
  • To make the CLI with Typer runnable. I used a modified method of this tutorial from T.Stringer but mainly (maybe only) the setup.py.

Formatter in VSCode#

Ellipsis object in Python#

Pylance typing#

  • @2022-01-22
  • Stub files are in this repo: python/typeshed
  • Work: PR#6998 on python/typeshed
  • Trace: While doing Selenium, the selenium.webdriver.chrome.service is not correctly defined. The Pylint reports an error "No parameter named "service" and I was trying to create an issue in Selenium repo, but the definition there was correct. Then I checked file path and found it is from vscode-pylance, knowing Pylint is based on Pylance. But the official repo microsoft/pylance-release doesn't contain any stub files. Its README file redirects (in first paragraph, though I saw it from the las paragraph) to microsoft/Pyright and then linked to python/typeshed.

Run as module#

Async Timer#

Selenium4 Chrome Profile#

Poetry#

  • @2022-01-31
  • Install: curl -sSL https://install.python-poetry.org | python3 - From Poetry Official Doc
  • Venv in project poetry config virtualenvs.in-project true From Poetry Official Doc: venv and Poetry Official Doc: venv in project
  • @2022-02-01
  • Publish: poetry publish From Poetry Official Doc: CLI publish, but need to set PyPI account with either (username and password) or (token). The token is only shown once after getting it.
  • URL: Poetry Official Doc: URLs can be used to publish to PyPI.
  • Poetry will add the developing package to the venv. Removing all packages with pip will cause an error (cannot use the CLI in tool.poetry.scripts) that I cannot fix, including rebuilding the venv. Work-around is to restart a project with poetry new.

Codecs#

  • @2022-01-31
  • The default binary codes with str/repr is shown in latin. Inspired by Python: Converting from ISO-8859-1/latin1 to UTF-8
  • Encoding base64 can represent all bytes in string exactly, latin/latin-1 will have some hidden characters (like \x00 and \x010).

Python Decorator#

Change suffix with pathlib#

PyNaCl#

  • @2022-02-01
  • PyNaCl is a Python binding for NaCl.
  • The NaCl name (Networking and Cryptography library) is interesting. also the cr.yp.to domain.
  • box = nacl.secret.SecretBox(key) uses a nonce which affects encryption. Yet decryption is not affected by the nonce.

Encoding#

  • @2022-02-02
  • Python encoding is a so huge library
  • Tried to escape latin characters with \x.

Pylint#

Remove file#

  • @2022-02-01
  • With pathlib: Path.unlink()
  • With os: os.remove("demo/test-del.txt")

Enum in Python#

PyPI#

  • @2022-02-02
  • The token is only shown once after getting it.

Remove all package#

Load JSON schema as Class#

Python Singleton#

Interesting new grammar#

Python Versioning#