Look before you leap

There are countless reasons to do unsafe memory access.  Here's some help:

import ctypes

import os

if hasattr(ctypes.cdll, "msvcrt"):  #windows branch

    _BUF = ctypes.create_string_buffer('a')

    def _check(address):

        try:

            ctypes.cdll.msvcrt.memcpy(_BUF, address, 1)

            return True

        except WindowsError:

            return False

else:

    libc = ctypes.CDLL('libc.so.6')

    libc.write.argtypes = ctypes.c_int, ctypes.c_void_p, ctypes.c_size_t

    _BUF = open('/tmp/_buf', 'w')

    _FD = _BUF.fileno()

    os.unlink(_BUF)

    def _check(address):

        rc = libc.write(_FD, address, 1)

        _BUF.seek(0)

        return rc == 1

def is_valid_ptr(address):

    'determines if the passed address (integer) is a valid pointer'

    return _check(address)

Loopy references

This may not come as a surprise to experienced Python programmers, but objects in Python can reference or "contain" themselves and/or their parent objects.

A trivial example, the self-containing list:

>>> a = []
>>> a.append(a)
>>> a
[[...]]

Kind of a cool repr, at least. How else could you flatly represent this infinitely nested object? What happens when you take two of these and smash them together with an equals comparator?

>>> b = []
>>> b.append(b)
>>> a == b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: maximum recursion depth exceeded in cmp

Like two mirrors pointed at one another, they compare back and forth forever, or at least for...

>>> sys.getrecursionlimit()
1000 

... loops.

collections.deque random access is O(n)

>>> r = range(int(1e6))
>>> li = list(r)
>>> de = collections.deque(r)
>>> pos = len(de) / 2
>>> timeit.timeit(lambda: de[100])
0.1355267132935296
>>> timeit.timeit(lambda: li[100])
0.14215966414053582

Great, list and deque performance are identical.

>>> timeit.timeit(lambda: li[pos])
0.1369839611421071
>>> timeit.timeit(lambda: de[pos])
56.227819584345696

Oops.  O(1) vs O(n).

necromancy, lesson one

There is a dark, mysterious, and dangerous power of Necromancy in the Python language; used only at great peril: the __del__ method.  Consider this class:

>>> class Living(object):
...    def __del__(self):
...       undead_hoard.append(self)
...
>>> undead_hoard = []

Let's use it.

>>> a = Living()
>>> undead_hoard
[]

So far so good.  What happens if we delete one?

>>> del a
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

The object got resurrected by __del__.  No problem, it can be removed from the list:

>>> del undead_hoard[0]
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Hmm, maybe reset the list?

>>> undead_hoard = []
>>> undead_hoard
[<__main__.Living object at 0x020C01D0>]

Lesson one of Python Necromancy: __del__ may be called many times on the same object; a safe __del__ must be re-entrant.  In this case, the resurrection was the fault of the __del__ method itself, but that is not always the case.

Who weakrefs the weakrefs?

>>> import weakref
>>> class Target(object): pass
...
>>> t = Target()
>>> r = weakref.ref(t)
>>> r2 = weakref.ref(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot create weak reference to 'weakref' object

decoding base64.decode

base64 is Python's module for providing various standard encoding functionalities, especially the eponymous base64. Little quirk about the decoding in Python 2.7, it ignores invalid characters!



>>> import base64
>>> test_str = base64.b64encode('123456789')
>>> base64.b64decode(test_str[:-4] + '....' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '...' + test_str[-4:])
'123456789'
>>> base64.b64decode(test_str[:-4] + '.' + test_str[-4:])
'123456789'
>>> test_str[:-4]
'MTIzNDU2'
>>> base64.b64decode(test_str[:-4])
'123456'

Python 3 adds a flag, validate, but it's off by default. Oh well!