Tech Glossary

From awctech wiki
Jump to: navigation, search

Continuous Integration, Delivery, Deployment

Mostly from this stackoverflow answer

  • Continuous Integration: practice of a dev merging code into master/baseline several to many times per day. Merge early and often, otherwise risk of integration conflicts is high. Small tasks allow for continuous integration, larger tasks would be frequent integration.
  • Continuous Delivery: part of Agile, everything required to implement automated verification of being done / QA'ed after a commit/merge. Good diagram
  • Continuous Deployment: move the result of dev process (QA'ed release-candidate) to production-like environment ready for functional testing. (This is not Continuous Release, a release may be strategic or political and need manual intervention.)

Functional Programming

From wikipedia:

In functional code, the output value of a function depends only on the arguments that are passed to the function, so calling a function f twice with the same value for an argument x produces the same result f(x) each time; this is in contrast to procedures depending on a local or global state, which may produce different results at different times when called with the same arguments but a different program state. Eliminating side effects, i.e., changes in state that do not depend on the function inputs, can make it much easier to understand and predict the behavior of a program, which is one of the key motivations for the development of functional programming.

Example languages include Common Lisp, Haskell, and Erlang. Scala provides support for functional programming. Blog post about functional programming vs OO

Python Specifics

Python Patterns

Map, Filter, Reduce

map: run function on each list element and return list of same size with altered elements

>>> mod_2 = map(lambda x: x%2, [0,1,2,3,4,5,6,7,8,9])
>>> list(mod_2)
[0, 1, 0, 1, 0, 1, 0, 1, 0, 1]

filter: run function on each list element, keep those where function resolves to True, return a filtered list

>>> less_than_5 = filter(lambda x: x<5, [0,1,2,3,4,5,6,7,8,9])
>>> list(less_than_5)
[0, 1, 2, 3, 4]

reduce: run function on first two list elements, and apply result sequentially for all elements until one value remains

>>> from functools import reduce
>>> reduce((lambda x, y: 2*x+y), [1,1,1])
7

Note: lambda can be used in other ways, from the python tutorial:

>>> def make_incrementor(n):
...     return lambda x: x + n
>>> make_incrementor(42)(1)
43
>>> f = make_incrementor(42)
>>> f(1)
43
>>> pairs = [(1, 'one'), (2, 'two'), (3, 'three'), (4, 'four')]
>>> pairs.sort(key=lambda pair: pair[1])

Fast ways to uniquify a list

>>> seq = 'hello there'
>>> list(set(seq))  # not order preserving
['l', 'e', 'h', ' ', 'o', 'r', 't']
>>> list(dict.fromkeys(seq))  # order preserving in python 3.6+
['h', 'e', 'l', 'o', ' ', 't', 'r']

Counting Patterns

>>> words = ['dog', 'cat', 'cat', 'lizard', 'dog', 'dog']

dictionary

>>> word_counts = dict()
>>> for w in words:
...     word_counts[w] = word_counts.get(w, 0) + 1
...
>>> word_counts
{'lizard': 1, 'dog': 3, 'cat': 2}

defaultdict - can pass any function: test = collections.defaultdict(lambda: [0] * 5) ... test['any key'] is [0, 0, 0, 0, 0]

>>> import collections
>>> word_counts = collections.defaultdict(int)
>>> # word_counts = collections.defaultdict(lambda: 0)  ## or this
>>> for w in words:
...     word_counts[w] += 1
...
>>> word_counts
defaultdict(<class 'int'>, {'lizard': 1, 'dog': 3, 'cat': 2}

Counter() - pass hashable sequence to get count of elements with element as dict key

>>> word_counts = collections.Counter(words)
>>> word_counts
Counter({'dog': 3, 'cat': 2, 'lizard': 1})
>>> word_counts.most_common(2)
[('dog', 3), ('cat', 2)]
>>> word_counts.keys()
dict_keys(['lizard', 'dog', 'cat'])

Python Iterables and Generators

Mostly from this article

  • Iterables are objects (i.e. lists, dicts, strings) where you can use iteration
  • Generators are iterables where iteration can happen only once. You can iterate with for loop or .next() method
>>> generator_var = (x*x for x in range(3))
>>> for i in generator_var: print(i)
0
1
4
>>> for i in generator_var: print(i)
>>>
  • Yield is used like return but will return a generator object. The function behind a generator object will not run until generator is iterated over.
>>> def generator_func():
...     for x in range(3):
...         yield x**2
>>> mygen = generator_func()  # body of function is not run with this statement
>>> # each iteration or next() causes the loop in the function to run
>>> for value in mygen:
...     print(value)
...     break
0
>>> next(mygen)
1
>>> next(mygen, 'optional default return')
4
>>> next(mygen, 'optional default return')
'optional default return'

Python Argument Values

Good points of consideration from the official Python Tutorial:

Default argument values

The default values are evaluated at the point of function definition in the defining scope, so that

>>> i = 5
>>> def f(arg=i):
...     print(arg)
>>> i = 6
>>> f()
5

Important warning: The default value is evaluated only once. This makes a difference when the default is a mutable object such as a list, dictionary, or instances of most classes. For example, the following function accumulates the arguments passed to it on subsequent calls:

>>> def f(a, L=[]):
...     L.append(a)
...     return L
>>> print(f(1))
[1]
>>> print(f(2))
[1, 2]
>>> print(f(3))
[1, 2, 3]

Functions definitions using arguments

*name and **name can accept a variable length number of positional and keyword arguments, respectively. Function receives a tuple and dictionary respectively.

*name and **name can also be used to unpack arguments:

...the built-in range() function expects separate start and stop arguments. If they are not available separately, write the function call with the *-operator to unpack the arguments out of a list or tuple:

>>> list(range(3, 6))            # normal call with separate arguments
[3, 4, 5]
>>> args = [3, 6]
>>> list(range(*args))            # call with arguments unpacked from a list
[3, 4, 5]

Note the * operator can be used with zip() to unzip a list, from the docs:

>>> x = [1, 2, 3]
>>> y = [4, 5, 6]
>>> zipped = zip(x, y)
>>> list(zipped)
[(1, 4), (2, 5), (3, 6)]
>>> x2, y2 = zip(*zip(x, y))
>>> x == list(x2) and y == list(y2)
True

Run Manual Unittest

Run unittest library module as a script using -m. Can run just one method or file/class

python -m unittest test_folder.test_file.TestCaseClass.test_method
python -m unittest tests.test_scraper.ScraperTestCase.test_magazine_scrape