HTTP download with dynamic reallocation and retries

at first StartDownload use recursion, when it absolute not need here. enough simply loop like -

ULONG nTry = 4, dwError;
do {
    dwError = Download();
} while(dwError && --Try);

next - you not need recreate all handles - hSession, hConnect, hRequest on send request fail, but only hRequest. the hSession we usual must open only once. the hConnect - once per url. note that WinHttpConnect really not connect to server. it simply remember server url and port in internal structures. if WinHttpOpen or WinHttpConnect - no sense try call it again (this is offline calls)

so code for synchronous (which is bad) processing must look like:

void test()
{
    if (HINTERNET hSession = WinHttpOpen( 0,
        WINHTTP_ACCESS_TYPE_NO_PROXY,
        WINHTTP_NO_PROXY_NAME,
        WINHTTP_NO_PROXY_BYPASS, 0))
    {
        if (HINTERNET hConnect = WinHttpConnect(hSession, L"stackoverflow.com", INTERNET_DEFAULT_HTTPS_PORT, 0))
        {
            int nTry = 4;
            ULONG dwError;
            do
            {
                if (HINTERNET hRequest = WinHttpOpenRequest(hConnect, L"GET", NULL,
                    NULL, WINHTTP_NO_REFERER,
                    WINHTTP_DEFAULT_ACCEPT_TYPES,
                    WINHTTP_FLAG_REFRESH|WINHTTP_FLAG_SECURE))
                {
                    if (WinHttpSendRequest(hRequest,
                        WINHTTP_NO_ADDITIONAL_HEADERS,
                        0, WINHTTP_NO_REQUEST_DATA, 0,
                        0, 0) && WinHttpReceiveResponse(hRequest, 0))
                    {
                        dwError = Download(hRequest);
                    }
                    else
                    {
                        dwError = GetLastError();
                    }

                    WinHttpCloseHandle(hRequest);
                }
                else
                {
                    dwError = GetLastError();
                }

            } while (dwError && --nTry);

            WinHttpCloseHandle(hConnect);
        }

        WinHttpCloseHandle(hSession);
    }
}

now look for you Download implementation - it contains mistakes and amazing not effective.

VirtualAlloc(0, dwSize, MEM_COMMIT, PAGE_READWRITE);

you always, unconditionally allocate temporary memory buffer (buf), even if you main buffer jar containing enough free space. than reallocate jar if need and copy temporary buffer to it memcpy((jar + totalDownloaded), buf, dwDownloaded);

also note that you have potential memory leak here - WinHttpReadData fail - you just break loop, without free temporary buf.

really we not need allocate any temporary buffer. we need just read data to main buffer jar - WinHttpReadData(hRequest, (PBYTE)jar + totalDownloaded, dwSize, &dwSize). if jar not have enough free space - we need reallocate it before WinHttpReadData call.

now look how you reallocate jar - it's just inexplicable for me

realloc: ** goto realloc;

for what this loop ?!? why need curalloc = curalloc * 2; and multiple time allocate, copy, free data ?!?. when we can just set

curalloc = totalDownloaded + dwDownloaded;

more better of course allocate some more memory than totalDownloaded + dwDownloaded. so code for Download can look like:

ULONG Download(HINTERNET hRequest)
{
    ULONG curalloc = 0x10000, dwSize, totalDownloaded = 0, cbNeed;

    HANDLE hHeap = GetProcessHeap();

    ULONG dwError = NOERROR;

    if (PVOID jar = HeapAlloc(hHeap, 0, curalloc))
    {
        do
        {
            if (!WinHttpQueryDataAvailable(hRequest, &dwSize))
            {
                dwError = GetLastError();
                break;
            }

            cbNeed = totalDownloaded + dwSize;

            if (cbNeed > curalloc)
            {
                cbNeed = (cbNeed + 0xffff) & ~0xffff;// round to 64kb

                if (PVOID buf = HeapReAlloc(hHeap, 0, jar, cbNeed))
                {
                    curalloc = cbNeed, jar = buf;
                }
                else
                {
                    dwSize = ERROR_OUTOFMEMORY;
                    break;
                }
            }

            if (!WinHttpReadData(hRequest, (PBYTE)jar + totalDownloaded, dwSize, &dwSize))
            {
                dwError = GetLastError();
                break;
            }

            totalDownloaded += dwSize;

        } while (dwSize);

        if (!dwError)
        {
            DbgPrint("download %u Ok\n", totalDownloaded);
        }

        HeapFree(hHeap, 0, jar);
    }

    return GetLastError();
}

this is already ok, but not best. windows let reserve memory space. so good solution - at begin reserve large enough memory space (in 32bit we can reserve how minimum n*10Mb, in 64bit - gigabytes). and then allocate memory from this reserved range. with this we not need reallocate and copy data. class for this dynamic allocation:

class DynamicBuffer
{
    PBYTE _BaseAddress;
    SIZE_T _dwReserve, _dwSize, _dwCommit;

    static SIZE_T RoundSize(SIZE_T size)
    {
        static SIZE_T s_dwAllocationGranularity;

        if (!s_dwAllocationGranularity)
        {
            SYSTEM_INFO si;
            GetSystemInfo(&si);
            s_dwAllocationGranularity = si.dwAllocationGranularity - 1;
        }

        return (size + s_dwAllocationGranularity) & ~s_dwAllocationGranularity;
    }

public:

    DynamicBuffer()
    {
        _BaseAddress = 0, _dwReserve = 0, _dwSize = 0, _dwCommit = 0;
    }

    ~DynamicBuffer()
    {
        Reset();
    }

    ULONG Create(SIZE_T dwSize)
    {
        if (_BaseAddress = (PBYTE)VirtualAlloc(0, dwSize = RoundSize(dwSize), MEM_RESERVE, PAGE_READWRITE))
        {
            _dwReserve = dwSize;
            return NOERROR;
        }

        return GetLastError();
    }

    ULONG AllocBuffer(PVOID* ppv, SIZE_T cb)
    {
        if (_dwReserve - _dwSize < cb)
        {
            return ERROR_OUTOFMEMORY;
        }

        SIZE_T dwSize = _dwSize + cb;

        if (dwSize > _dwCommit)
        {
            SIZE_T dwCommit = RoundSize(dwSize);

            if (!VirtualAlloc(_BaseAddress + _dwCommit, dwCommit - _dwCommit, MEM_COMMIT, PAGE_READWRITE))
            {
                return GetLastError();
            }

            _dwCommit = dwCommit;
        }

        *ppv = _BaseAddress + _dwSize;

        return NOERROR;
    }

    void AddData(SIZE_T cb)
    {
        _dwSize += cb;

        if (_dwSize > _dwCommit)
        {
            __debugbreak();
        }
    }

    PVOID getData()
    {
        return _BaseAddress;
    }

    SIZE_T getDataSize()
    {
        return _dwSize;
    }

    SIZE_T getFreeSpace()
    {
        return _dwReserve - _dwSize;
    }

    void Reset()
    {
        if (_BaseAddress)
        {
            VirtualFree(_BaseAddress, 0, MEM_RELEASE);
            _BaseAddress = 0;
        }
        _dwReserve = 0, _dwSize = 0, _dwCommit = 0;
    }
};

with this we can rewrite Download :

ULONG Download(HINTERNET hRequest)
{
    DynamicBuffer buf;

    ULONG dwError = NOERROR;

    if (dwError = buf.Create(0x4000000)) // 64Mb reserve
    {
        return dwError;
    }

    ULONG dwSize;

    do
    {
        if (!WinHttpQueryDataAvailable(hRequest, &dwSize))
        {
            dwError = GetLastError();
            break;
        }

        PVOID pv;
        if (dwError = buf.AllocBuffer(&pv, dwSize))
        {
            break;
        }

        if (!WinHttpReadData(hRequest, pv, dwSize, &dwSize))
        {
            dwError = GetLastError();
            break;
        }

        buf.AddData(dwSize);

    } while (dwSize);

    if (!dwError)
    {
        DbgPrint("download %u Ok\n", buf.getDataSize());
    }

    return GetLastError();
}

but however always much better do asynchronous download. this can be done say in next way:

class __declspec(novtable) InternetHandle
{
    HINTERNET _hInternet;
    LONG _dwRef;

protected:

    virtual ~InternetHandle()
    {
        if (_hInternet)
        {
            WinHttpCloseHandle(_hInternet);
        }
        DbgPrint("%s<%p>\n", __FUNCTION__, this);
    }

    InternetHandle()
    {
        _dwRef = 1;
        _hInternet = 0;
        DbgPrint("%s<%p>\n", __FUNCTION__, this);
    }

public:

    void AddRef()
    {
        InterlockedIncrement(&_dwRef);
    }

    void Release()
    {
        if (!InterlockedDecrement(&_dwRef)) delete this;
    }

    HINTERNET get_handle() { return _hInternet; }

    HINTERNET set_handle(HINTERNET hInternet) {
        return InterlockedExchangePointer(&_hInternet, hInternet);
    }
};

class CSession : public InternetHandle
{
public:
    ULONG Open(LPCWSTR pszAgentW = 0)
    {
        if (HINTERNET hSession = WinHttpOpen( pszAgentW,
            WINHTTP_ACCESS_TYPE_NO_PROXY,
            WINHTTP_NO_PROXY_NAME,
            WINHTTP_NO_PROXY_BYPASS, WINHTTP_FLAG_ASYNC))
        {
            set_handle(hSession);

            return NOERROR;
        }

        return GetLastError();
    }
};

class CTarget : public InternetHandle
{
    CSession* _pSession;

    virtual ~CTarget()
    {
        _pSession->Release();
    }
public:

    ULONG Set(PCWSTR pswzServerName, INTERNET_PORT nServerPort)
    {
        if (HINTERNET hConnect = WinHttpConnect(_pSession->get_handle(), pswzServerName, nServerPort, 0))
        {
            set_handle(hConnect);
            return NOERROR;
        }

        return GetLastError();
    }

    CTarget(CSession* pSession) : _pSession(pSession)
    {
        pSession->AddRef();
    }
};

class DownloadCtx : public InternetHandle, public DynamicBuffer, public WINHTTP_ASYNC_RESULT
{
    CTarget* _pTarget;
    ULONG _nTryCount;
    ULONG _dwThreadId;

    virtual ~DownloadCtx()
    {
        DbgPrint("<%u,%u> %u\n", dwResult, dwError, getDataSize());

        _pTarget->Release();

        PostThreadMessage(_dwThreadId, WM_QUIT, dwResult, dwError);
    }

public:
    DownloadCtx(CTarget* pTarget, ULONG nTryCount) : _pTarget(pTarget), _nTryCount(nTryCount)
    {
        pTarget->AddRef();
        dwError = ERROR_IO_PENDING, dwResult = 0;
        _dwThreadId = GetCurrentThreadId();
    }

    ULONG SendRequest()
    {
        if (get_handle()) __debugbreak();

        if (!dwError)
        {
            return NOERROR;
        }

        if (!_nTryCount--)
        {
            return dwError;
        }

        dwError = ERROR_IO_PENDING, dwResult = 0;

        Reset();

        DbgPrint("=========== %x ============\n", _nTryCount);

        if (HINTERNET hRequest = WinHttpOpenRequest(_pTarget->get_handle(), L"GET", NULL,
            NULL, WINHTTP_NO_REFERER,
            WINHTTP_DEFAULT_ACCEPT_TYPES,
            WINHTTP_FLAG_REFRESH|WINHTTP_FLAG_SECURE))
        {
            PVOID Context = this;

            if (WinHttpSetOption(hRequest, WINHTTP_OPTION_CONTEXT_VALUE, &Context, sizeof(Context)))
            {
                AddRef();

                set_handle(hRequest);

                WinHttpSetStatusCallback(
                    hRequest,
                    _StatusCallback,
                    WINHTTP_CALLBACK_FLAG_ALL_NOTIFICATIONS,
                    NULL );

                if (!WinHttpSendRequest(hRequest,
                    WINHTTP_NO_ADDITIONAL_HEADERS,
                    0, WINHTTP_NO_REQUEST_DATA, 0,
                    0, (DWORD_PTR)this))
                {
                    Close(GetLastError(), API_SEND_REQUEST);
                }

                return NOERROR;
            }

            WinHttpCloseHandle(hRequest);
        }

        return GetLastError();
    }

    ULONG Read(HINTERNET hRequest)
    {
        if (DWORD dwNumberOfBytesToRead = (DWORD)min(0x10000, getFreeSpace()))
        {
            PVOID Buf;
            if (ULONG err = AllocBuffer(&Buf, dwNumberOfBytesToRead))
            {
                return err;
            }

            DbgPrint("Begin Read %u\n", dwNumberOfBytesToRead);

            return WinHttpReadData(hRequest, Buf, 0x10000, 0) ? NOERROR : GetLastError();
        }

        return ERROR_BUFFER_OVERFLOW;
    }

    void Close(ULONG err, DWORD_PTR from)
    {
        if (HINTERNET hRequest = set_handle(0))
        {
            dwError = err, dwResult = from;
            WinHttpCloseHandle(hRequest);
        }
    }

    static void CALLBACK _StatusCallback(
        __in  HINTERNET hRequest,
        __in  DWORD_PTR Context,
        __in  DWORD dwInternetStatus,
        __in  LPVOID lpvStatusInformation,
        __in  DWORD dwStatusInformationLength
        )
    {
        reinterpret_cast<DownloadCtx*>(Context)->StatusCallback(
            hRequest, dwInternetStatus, lpvStatusInformation, dwStatusInformationLength);
    }

    void StatusCallback(
        __in  HINTERNET hRequest,
        __in  DWORD dwInternetStatus,
        __in  LPVOID lpvStatusInformation,
        __in  DWORD dwStatusInformationLength
        );
};

void DownloadCtx::StatusCallback(
                                      __in  HINTERNET hRequest,
                                      __in  DWORD dwInternetStatus,
                                      __in  LPVOID lpvStatusInformation,
                                      __in  DWORD dwStatusInformationLength
                                      )
{
    DbgPrint("%x>%p %08x %p %x\n", GetCurrentThreadId(), hRequest,
        dwInternetStatus, lpvStatusInformation, dwStatusInformationLength);

    ULONG Error;
    DWORD_PTR Result;

    switch (dwInternetStatus)
    {
    default: return;

    case WINHTTP_CALLBACK_STATUS_HANDLE_CLOSING:
        SendRequest();
        Release();
        return;

    case WINHTTP_CALLBACK_STATUS_REQUEST_ERROR:
        Error = reinterpret_cast<WINHTTP_ASYNC_RESULT*>(lpvStatusInformation)->dwError;
        Result = reinterpret_cast<WINHTTP_ASYNC_RESULT*>(lpvStatusInformation)->dwResult;
        break;

    case WINHTTP_CALLBACK_STATUS_SENDREQUEST_COMPLETE:
        if (WinHttpReceiveResponse(hRequest, NULL))
        {
            return;
        }

        Error = GetLastError(), Result = API_RECEIVE_RESPONSE;
        break;

    case WINHTTP_CALLBACK_STATUS_READ_COMPLETE:

        DbgPrint("READ_COMPLETE:%u\n",dwStatusInformationLength);

        if (dwStatusInformationLength)
        {
            AddData(dwStatusInformationLength);

            if (Error = Read(hRequest))
            {
                Result = API_READ_DATA;
                break;
            }
            return ;
        }

        Error = NOERROR, Result = 0;
        break;

    case WINHTTP_CALLBACK_STATUS_HEADERS_AVAILABLE:

        static volatile UCHAR guz;
        union {
            PWSTR sz;
            PVOID buf;
        };
        PVOID stack = _alloca(guz);
        ULONG cb = 0, rcb = 32;
        do
        {
            if (cb < rcb)
            {
                rcb = cb = (ULONG)((PBYTE)stack - (PBYTE)(buf = _alloca(rcb - cb)));
            }

            if (WinHttpQueryHeaders(hRequest, WINHTTP_QUERY_CONTENT_LENGTH, 0, buf, &rcb, 0))
            {
                Error = NOERROR;

                if (cb = wcstoul(sz, &sz, 10))
                {
                    DbgPrint("Content-Length: %u\n", cb);
                    cb++;
                }
                else
                {
                    Error = ERROR_NO_DATA;
                }
                break;
            }
        } while ((Error = GetLastError()) == ERROR_INSUFFICIENT_BUFFER);

        Result = 0;

        switch (Error)
        {
        case ERROR_WINHTTP_HEADER_NOT_FOUND:
            cb = 0x4000000;// 64 mb reserve
        case NOERROR:
            if (Error = Create(cb))
            {
                break;
            }

            if (Error = Read(hRequest))
            {
                Result = API_READ_DATA;
                break;
            }

            return ;
        }
        break;
    }

    Close(Error, Result);
}

void winhttp()
{
    if (CSession* pSession = new CSession)
    {
        if (!pSession->Open())
        {
            if (CTarget* pTarget = new CTarget(pSession))
            {
                if (!pTarget->Set(L"stackoverflow.com", INTERNET_DEFAULT_HTTPS_PORT))
                {
                    if (DownloadCtx* Ctx = new DownloadCtx(pTarget, 4))
                    {
                        Ctx->SendRequest();

                        Ctx->Release();
                    }
                }
                pTarget->Release();
            }
        }

        pSession->Release();
    }

    MessageBoxW(0,0,L"for demo",0);
}


Throwing the contents into a dictionary will lose any sorting you may have had. Dictionaries are unordered collections.

The cleanest way to deal with this really is to implement your own custom comparer to compare the columns and reset their order based on the ordering.

I believe this would be more or less logically equivalent:

public class MyColumnComparer : Comparer<DataColumn>
{
    static Regex re = new Regex(@"[^\(\)][^\(]*\((\d{2}/\d{4})\).*");
    public override int Compare(DataColumn x, DataColumn y)
    {
        // a regular expression might be overkill here but it will be easier to maintain
        var xMatch = re.Match(x.Caption);
        var yMatch = re.Match(y.Caption);

        // both did not contain the pair
        if (!xMatch.Success && !yMatch.Success)
        {
            // use original ordering
            return x.Ordinal.CompareTo(y.Ordinal);
        }

        // one contained the pair
        if (xMatch.Success != yMatch.Success)
        {
            return yMatch.Success ? -1 : 1;
        }

        // both contains a pair
        var xDate = DateTime.ParseExact(xMatch.Groups[1].Value, "MM/yyyy",
            CultureInfo.InvariantCulture);
        var yDate = DateTime.ParseExact(yMatch.Groups[1].Value, "MM/yyyy",
            CultureInfo.InvariantCulture);
        return xDate.CompareTo(yDate);
    }
}

static void ProcessTable(DataTable table)
{
    var sortedColumns = table.Columns
        .Cast<DataColumn>()
        .OrderBy(col => col, new MyColumnComparer())
        .ToList();
    var index = 0;
    foreach (var column in sortedColumns)
        column.SetOrdinal(index++);
}


You should first ask yourself this: Why am I sanitizing input in this way?

Let's say I have a login form, and the user enters his credentials, an email and a password. The email will be queried against the database, to figure out the user's hashed password (You are hashing your passwords, right?!), and the password would be hashed, to be compared against that hash.

Both of those fields will never be outputted to HTML, the password is never kept in plain-text form for very long, and the username is queried against the database and then discarded. So why should you escape them for HTML?

Sanitizing (or more accurately named, Escaping), contrary to popular belief should not be done when you receive your input. It should be done right before you output it. Because at the time you receive your input, you have no idea where it's going to be used. It could be outputted to HTML (in which case it needs htmlspecialchars()), it could be outputted to JSON for an AJAX response (in which case it needs json_encode()), it could be stored in the database (in which case it needs prepared statements).

The general rule is: You escape as late as possible. HTML strings should be escaped right before they get echoed onto an HTML page.

What you should be doing when receiving your input, and as early as possible, is validating the input, i.e. making sure the input matched what you expect in format, and in business logic. Format means "This is a valid email address", and business logic means "This comment that the user is trying to edit, really belongs to him".


How do you load the data for events and people? If you can denormalise your data so that a person has a materialised Set/List of events they are involved in I suspect you would gain a lot of performance (at the cost of a bit of memory).

Here is some pseudo code to explain what I mean:

var personStore = new Dictionary<string, Person>();
var eventStore = new Dictionary<string, Event>();
foreach (var eventData in eventSource)
{
  var event = new Event(eventData);
  eventStore[eventData.Name] = event;

  foreach (var personData in eventData)
  {
    // Is this the first time we've seen this person?
    if (!personStore.HasKey(personData.Name)
    {
      personStore[personData.Name] = new Person(personData);
    }

    var person = personStore[personData.Name];

    // This creates the bidirectional mapping which makes lookups
    // in either direction fast.
    event.PeopleInvolved.Add(person);
    person.Events.Add(event);
  }
}


A function that finds the kth smallest integer in array A (unsorted), of size n

The compiler is your friend

Activate all warnings and you'll get pretty good hints :

selection.c: In function ‘kth_smallest_number’:
selection.c:17:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^

Test your code

Here's what I have written to test your code :

int A[] = {4,4,56,78,54,6512,21,8,7,2,321,87,78,54,2,21,5,4,321,456,5,4,35,5,4,1,67,8,1,54,4,654,7,6,54,321,7,32,4,21,54,7};
int n = sizeof(A)/sizeof(A[0]);
for (int k=0; k<n; k++)
{
    printf("%d %d\n",k,kth_smallest_number(A,n,k));
}

I'd expect the returned value to be sorted and it's clearly not the case here. Thus, I expect something to be wrong.

Playing with other inputs : int A[] = {10,9,8,7,6,5,4,3,2,1,0}; , int A[] = {10,9,8,7,6,5,5,4,3,2,1,0};, it seems like your solution does not handle properly duplicated value.

A matter of style

The indentation looks weird on your question.

You should try to declare your local variable in the smallest possible scope. Among other things, you can define your loop indices in the loop syntax in you use the C99 mode (-std=c99).

Once rewritten (but still wrong), your code looks like :

int kth_smallest_number(int A[], int n, int k){
    for(int i=0; i<n; i++){
        /*Check whether the number A[i] satisfies
          the condition*/
        int smaller_than_count = 0;
        for(int j=0; j<n; j++){
            if(A[i]<=A[j]){
                smaller_than_count++;
            }
        }
        if(smaller_than_count == n-k){
            return A[i];
        }
    }
}

Edit :

An attempt to correctness

Here what I have written , it seems to work fine but I won't promise anything.

int kth_smallest_number(int A[], int n, int k){
    for(int i=0; i<n; i++){
        /*Check whether the number A[i] satisfies the condition*/
        int ai = A[i];
        int smaller_than_count = 0;
        int bigger_than_count = 0;
        for(int j=0; j<n; j++){
            int aj = A[j];
            if (ai < aj) {
                smaller_than_count++;
                if (smaller_than_count >= n-k) break; // Optimisation
            } else if (ai > aj) {
                bigger_than_count++;
                if (bigger_than_count > k) break; // Optimisation
            }
        }
        if(smaller_than_count < n-k && bigger_than_count <= k){
            return ai;
        }
    }
    return -1;
}


Print a "shebang" line, or prepend it to a file

Is the code readable? Can this code be simpler?

Basically readable, although I prefer early returns and things like that.

Ported to bash but testing limited:

# shebang [-iv] [-t interpreter] [-I extension] [-J [extension]] [filename]
#
# If no filename is given, print a shebang. If a filename is given, append a
# shebang to the file and print its contents. If a filename is given and the -I
# or -J options are specified, append a shebang to the file in place.
#
# OPTIONS
#   -i
#    Interactive mode; ask for confirmation first.
#   -I
#    Modify "filename" in place (as in `sed -i`) using specified extension.
#   Extension is mandatory.
#   -J
#    Same as -I, but extension is mandatory only if the string "gsed" cannot
#    be found in the output of `which sed`. Overridden by -I.
#   -t
#    Specify an interpreter, e.g. `shebang -t zsh` produces "#!/usr/bin/env zsh"
#   -v
#    Print (to stderr) the interpreter being used.

shebang () {
    local inplace inplace2 verbose interactive
    local interpreter input_shebang shebang continue
    local sed_command gsed_avail _nextopt _line _readbool
    local OPT OPTIND OPTARG
    # HACK: shift manually when encounters J
    while getopts 't:I:viJ' OPT; do case "$OPT" in
        i)  interactive=1;;
        I)  inplace="$OPTARG";;
        J)  eval "nextopt=\$$OPTIND";
            case "$_nextopt" in
                -*) inplace2='-J';;
                *)  inplace2="-J $_nextopt"; shift;;
            esac;;
        t)  interpreter="$OPTARG";;
        v)  verbose=1;;
    esac; done
    shift $((OPTIND-1))

    (( $? )) && return 1

    if [[ -n $1 ]]; then
        IFS='' read -r _line < "$1"
        if (( $? )); then
            echo "Unable to read $1." >&2
            return 1
        fi
        # if echo "$_line" | grep ^#!; then
        if [[ "$_line" =~ ^\#! ]]; then
            echo "$1 already has a shebang." >&2
            return 1
        fi
    fi

    if [[ (-z "$interpreter") && (-z "$1") ]]; then
          echo "The -t option is mandatory if no argument is supplied." >&2
          return 1
    fi

    if [[ (-z "$interpreter") && (-n "$1") ]]; then
          interpreter=$(filename=$(basename "$1"); [[ "$filename" = *.* ]] && echo "${filename##*.}")  # grab extension
          interpreter=${interpreter/#* *}  # "extensions" with whitespace probably aren't legit
          : ${interpreter:=sh}  # assume sh if no extension and no -t option
    fi

    if [[ "$ZSH_VERION" ]]; then
        # reads one character.
        _sb_read1(){ read -k 1 "$@"; }
    else
        # ksh/bash
        _sb_read1(){ read -n 1 "$@"; }
    fi

    shebang="#!/usr/bin/env $interpreter"

    if [[ -n $verbose ]]; then
        echo "Using interpreter '$interpreter'" >&2
    fi

    if [[ -n $interactive ]]; then
        while true; do
            printf "continue? Shebang will be '$shebang'. Ok? y/n: "
            # ah, zsh/bash have different ideas on read.
            _sb_read1 _readbool
            echo
            case "$_readbool" in
                [nN])   return 1;;
                [yY])   break;;
            esac
        done
    fi

    if [[ -z "$1" ]]; then
          echo $shebang
          return 0
    fi

    if type gsed; then
        sed_command="1 i\\$shebang\n"
        if [[ -n $inplace ]]; then
            gsed "${inplace/-I/-i}" "$sed_command" "$1"
        elif [[ -n $inplace2 ]]; then
            gsed "${inplace2/-J/-i}" "$sed_command" "$1"
        else
            gsed "$sed_command" "$1"
        fi
    else
        sed_command=$'1i\n'$shebang'\n'
        if [[ -n $inplace ]]; then
            sed "${inplace/-I/-i}" "$sed_command" "$1"
        elif [[ (-n $inplace2) && (-z ${inplace2#-J}) ]]; then
            echo "'-J' was given without an argument, but 'gsed' is unavailable. Specify an explicit extension or use -I."
        elif [[ -n ${inplace2#-J} ]]; then
            sed "${inplace2/-J/-i}" "$sed_command" "$1"
        else
            sed "$sed_command" "$1"
        fi
    fi
}

Using that grep for if gives extra ksh compatibility (untested).


I don't have a C# compiler at hand and cannot test the following.

It looks like the Search() always returns the sum of results of these 3 things:

delegate the call to any property that's ISearchable.
search the value of some properties
do not search the value of some other properties

The method will likely always consist of a list of these types of code blocks:

the delegate type

foreach (var searchResult in this.PROPERTY_THAT_IS_ISEARCHABLE.Search(searchString))
{
    yield return searchResult;
}

for the direct search

yield return this.Search(searchString, this.GetType().GetProperty(nameof(this.PROPERTY_THATS_NOT_ISEARCHABLE)));

no code for excluded properties

This smells like copy&paste. But you are already doing reflection, so why not use it to your advantage?

I think you can create a generic Search() method, then slap that onto everything that implements ISearchable via extension, getting you rid of explicitely implementing the method in those classes. Here's how:

You should be able to filter out all properties that implement ISearchable and delegate the method call to them.
For non-ISearchable properties that should be searched, you should decide if they are the minority1. If so, define an attribute to opt-in a property. You'd then ignore all non-ISearchable by default and only search the values of those that are explicitly opted-in with the attribute.
For non-ISearchable properties that should not be searched, do pretty much the opposite of (2.). If they are the minority, create and add to them an attribute explicitly to opt-out of the default which would be to search all non-ISearchable properties.

1 The minority thing is only to reduce future work, minimizing the amount of attributes that you have to add. Attributes are optional, so you have to define some default behaviour anyway.

Let's say the default behaviour is to search all properties and opt-out via attribute. Here's how your classes could look like:

public class Book : ISearchable
{
    public DescriptionInfo DescriptionInfo { get; set; }
    public string Title { get; set; }
}

As you can see in the Book class: ISearchable is more or less a marker interface.

public class DescriptionInfo : ISearchable
{
    public string Description { get; set; }

    [IgnoreThisDuringSearch]
    public string ShortDescription { get; private set; }

    public DescriptionInfo(string description)
    {
        if (!string.IsNullOrEmpty(description))
        {
            this.Description = this.ShortDescription = description;
        }
    }
}

The generic extension method would perform the following two steps then:

delegate to every property that it can be delegated to
search every property that got opt-in via attribute or search every property except those opt-out via attribute

Some additional thoughts:

The attribute makes the search independent from the actual names of the properties. If you change Title to Totle in the Book class, you current code breaks. The suggested solution is solely based on reflection and existance (lack of) an attribute, which should always work, given a sensible default behaviour.

If you wish to perform the searching/delegating in a special order, you could do that, too. For example via a property of another attribute. See [DataMember(Order = n)] for example.


Using SQL to reorder items in a list view - is this efficient?

Enum
First of all, instead of an int orderByType, I would use an enum.
enum SortOrder {
    NEWER("date(" + SomeConstant.KEY_DATE + ") desc"),
    OLDER("date(" + SomeConstant.KEY_DATE + ") asc"),
    ...;
    private SortOrder(String order) {
        this.order = order;
    }

    public String getOrder() {
        return this.order;
    }
}

This will clean up your getAllTransactions method:
public Cursor getAllTransactions(SortOrder orderByType) {
    return getAllTransactions(orderByType == null ? null : orderByType.getOrder());
}

Database vs. Application
I feel that is OK to let the database deal with this. There is a positive aspect of letting it be handled in the application: If the application deals with it, then you can reorder the elements without querying the database.
However,

The list is small (5 records at the most)

I think either way will work just fine. If it is common to reorder the list, then I would let the application deal with it. Although your enum SortOrder needs quite a lot of more code to work directly with a List<T>.
If you want to use an enum and reorder inside the application, then I would let the enum take a Comparator<T> in the constructor and create an anonymous inner class for each enum value.


Scaling and cropping an image adaptively based on the window dimensions

This seems like the kind of thing that should be done purely in CSS using media queries, specifically orientation.  When the window is taller than it is wide, the rules inside

@media all and (orientation: portrait) { … }

… would apply.  When the window is wider than it is tall, it would be

@media all and (orientation: landscape) { ... }

Furthermore, I would consider that image to be background-image styling for
The post title
 rather than an  element in its own right.  The scaling should be handled by a background-size rule.


Have a look at TableView's datasource protocol:

– tableView:cellForRowAtIndexPath:
– numberOfSectionsInTableView:
– tableView:numberOfRowsInSection:
– sectionIndexTitlesForTableView:

I find it quite hard to come up with nice names for such sidle parameter method signature and code completion would be easier, if the form, you propose would exists, but if a method accepts parameters, it name must terminate with one of therm — if it just have one, it must be at the end.

Your idea of adding a dead parameter is IMO very bad design. Another parameter should lead possibly to another behavior. Also it would work against objective-c strength of self explaining method name. countOfSubscribersForNewsEmitter:(MyNewsEmitter *)emitter make sense, while newsEmitter:(MyNewsEmitter *)emitter countOfSubscribers:(BOOL) aBool does not make much sense, as the bool disallows us to form a correct mental model of the purpose of this method.


Not bad a all for a first Python program:

Good use of the line: if __name__ == '__main__': .
Good use of string methods (replace, isupper, islower etc...).
Good use of the random module methods.

But some things can be made better:

The function passwordgenerator could have pw_length as a parameter and return mypw. (The printing of the password moves to the main part in this case)
calling passwordgenerator from within passwordgenerator itself (a recursive call) can be tricky (what is the password accidentally is always lower...?)
The string constants "!?@%#$" and "abc...XYZ" occur both twice, they could be made constants on top of the program.
I think the line: elif count == pw_length should be different
a docstring could be added to passwordgenerator explaining the properties of the generated password.


Java probability if 2 integers are coprime

This review is for the code only, not the optimization / correctness; I assume the code works as IS.
Avoid using C-style array declaration
In the main method, you declared a C-style array declaration with the args variable.
before
String args[]

after
String[] args

In my opinion, this style is less used and can cause confusion.
Always add curly braces to loop & if
In my opinion, it's a bad practice to have a block of code not surrounded by curly braces; I saw so many bugs in my career related to that, if you forget to add the braces when adding code, you break the logic / semantic of the code.
Before
if (b == 0)
   return a;
else
   return myTestMethod(b, a % b);

After
if (b == 0) {
   return a;
} else {
   return myTestMethod(b, a % b);
}

Other things
To save space in the main method, you can inline the variable y and p
//[...]
int y = (myTestMethod(a, b));
//[...]
float p = (myTestMethod(a, b));
//[...]

You can take less lines for the printing in the main method
You can make a printing method
public static void main(String[] args) {
   if (y == 1) {  //if gcd is 1 the probability should be 1
      printMultiLine("probability is:", p);
   } else { //else print probability as p/100.
      printMultiLine("The probability is :", p / 100);
   }
}

private static void printMultiLine(String message, float value) {
   System.out.println(message);
   System.out.println(value);
}

or
You can uses java.io.PrintStream#printf
java.io.PrintStream#printf offer you to use patterns to build the string without concatenating it manually. The only downside is you will be forced to add the break line character yourself; in java you can use the %n to break the line (portable between various platforms) or uses the traditional \n / \r\n.
public static void main(String[] args) {
   //[...]
   if (y == 1) {  //if gcd is 1 the probability should be 1
      System.out.printf("probability is:%n%f", p);
   } else { //else print probability as p/100.
      System.out.printf("The probability is:%n%f", p / 100);
   }
}


for i in range(1,499): #max value for any side of the triangle must be less      than the semiperimeter
    for j in range(i,499):
        for k in range(j,499):
            if(i+j+k==1000 and i**2+j**2==k**2):

There are a couple quick optimizations.

for i in range(1,333):

You can restrict this beyond 499.  We know that \$ a < b < c\$ and \$a + b + c = 1000\$, so we can say that \$3*a < 1000\$ or \$a \leq 333\$.

    for j in range(i,499): #max value for any side of the triangle must be less than the semiperimeter
        k = 1000 - i - j

We don't need to iterate to find k.  We can calculate it from i and j since we know that the three sum to 1000.

        if (i**2+j**2==k**2):

So we can just check that it's a Pythagorean triple.  The sum will always be correct, as that's how we select k.

Just calculating k instead of iterating to find it should reduce the time to less than a second.


Spaces

Put a space after for and if:

for(int i = 0; i < bound; i++) --> for (int i = 0; i < bound; i++)
if(giveOutput) --> if (giveOutput)

Put a space around a -:

sample.length-1 --> sample.length - 1

And the same goes for +:

System.out.println("Average time taken to sort array: "+totalTime/bound+" nanoseconds");

becomes:

System.out.println("Average time taken to sort array: " + totalTime/bound + " nanoseconds");

if-statements

Add braces around if-statements:

if (giveOutput) {
    System.out.println(Arrays.toString(sample));
}

Avoid nested if-statements such as:

if(checkCorrectness)
    if(!isSorted(sample))
        System.out.println("WARNING! mistake detected");

Change that to:

if (checkCorrectness && !isSorted(sample)) {
    System.out.println("WARNING! mistake detected");
}

(Consider throwing an exception for this by the way. A mistake is important stuff)

Naming

Whatever qs is doing (A quicksort I guess), the name is too short to be meaningful. Name it quicksort instead.

Name your methods for inputting things as input*:

getNumber --> inputNumber
yes_no --> inputBoolean

  Are there any guidelines to follow for when prompting the user for information? One thing I'm not sure about is when to prompt with a question mark vs a colon (e.g. "what is x?" vs "input x:").

I believe this can be answered better on User Experience. They probably have a question about it already.

I would think that a question mark should be used when asking a question, and a colon should be used when encouraging the user to enter something. Of course this doesn't say much though as all your user prompts can be modified to the other form. My opinion is that what you currently have is good though.

Other comments

Good choice to use System.nanoTime();
To make the existence of another line in your output clear, I would change System.out.println(Arrays.toString(sample)+"\n"); to:

if (giveOutput) {
    System.out.println(Arrays.toString(sample));
    System.out.println();
}

In Java, the convention is that { goes on the same line, not on it's own line.

public static int getNumber(String message)
{

should be, according to the conventions:

public static int getNumber(String message) {

Avoid multiple variable declarations on the same line

long totalTime = 0, sumArraySizes = 0;

can be:

long totalTime = 0;
long sumArraySizes = 0;


Function "between" for "Everything in One Line" library

There's no need to use the conditional operator here. This would be enough:

bool between (WORD n, WORD x, WORD y) {
    return n >= x && n <= y;
}

Other than this, the variable names are not descriptive. It is not clear that x specifies the lower bound and y the higher bound.

bool between (WORD value, WORD low, WORD high) {
    return value >= low && value <= high;
}

Additionally, I have to question a little bit the usage for a "Everything in One Line" library. If the code that you are writing is trivial one-liners, it might be easier for other programmers to quickly re-write the actual code than to use your library. Most of the times I could use a between function, I would probably write the actual code in-line, without using an additional function for it at all. However, writing a "Everything in One Line"-library can of course be an educating experience for yourself.


BitArray (compacting 32 bool values into one integer)

public void set(int index, boolean value) {
    int actualIndex = (int)Math.floor(index / 32f);
    int elementValue = elements[actualIndex] & ~(0x01 << (index % 32));

    if (value) {
        elementValue = elementValue | (0x01 << (index % 32));
    }

    elements[actualIndex] = elementValue;
}

You can avoid a bit of recalculation here by caching the shift value. Also, switch to long to increase the range of values the user can enter

public void set(long index, boolean value) {
    final long actualIndex = (long)Math.floor(index / 32f);
    final long mask = 0x01 << (index % 32);
    long elementValue = elements[actualIndex] & ~mask;

    if (value) {
        elementValue |= mask;
    }

    elements[actualIndex] = elementValue;
}


Gravity simulation using Numpy and Pygame

The positions of the bodies are modified in-place, while the loop is running. It means that in your universe the action is not equal to reaction. Your simulation will exhibit some nonrealistic behaviors, like drift of the center of the mass.
Euler integration method is not the most accurate. Consider Runge-Kutta. At least, monitor the motion invariants (total energy, momentum, and angular momentum of the ensemble), and do corrective actions when they start diverging.
After f = m[i] * ... and f_total / m[i] the mass cancels. Multiplication and division could be safely omitted.
Along the same line, scale the masses by G once, before the simulation begins.


Command-Line tool to find large files on a Linux system

Some observations:

Use a main function and the argparse module pull out an argument parsing method to make this scriptable.
"larg" should be "large"; getsize should be get_size, fpath should be file_path. Saving characters costs a lot of money in maintainability.
As @Edward wrote, don't keep the entire list of files in memory. As a counter-point I would have liked for the results limit to not exist - the default behaviour would then be simpler to implement, and would return every file in order of size.
In addition to excluding directories you should make sure that you don't follow symbolic links, or you could end up with an infinite loop.
The OSError in this case should be fatal - it indicates that your program is faulty since it doesn't properly filter files which you can't get the size of. I believe it could also happen if a file is not accessible, which means any result you get from this script might be inaccurate.
Replace **kwargs with excludes=None, followed by a simple excludes = [] if excludes is None else excludes. That way the parameter list is unambiguous. Even easier, make excludes mandatory (you are always using it anyway). YAGNI & KISS apply :)
It looks like your handling of exclusions doesn't anchor the regex at the start of the filename, so you'll end up excluding for example /home/user/dev.
excludes in walk is always truthy. You should either remove the or r'$.' bit (best option IMO) or get rid of if excludes:.
You reuse excludes in walk to create a completely different list. This is a bad practice because it makes the use and content of a variable really hard to follow. Use separate variables for these.


I have some suggestions about PremieraTcpClient. The way it is written may lead to unreleased resources. If you have an error then you will remain with a stream and client  opened.
The correct way to do it is by using try...catch...finally or by using using.
Below you can find the code using try catch finally

    private string Connect()
    {
        string responseData = string.Empty;
        TcpClient client = null;
        NetworkStream stream = null;
        try
        {
            client = new TcpClient(ServerIp, Port);
            client.ReceiveBufferSize = Int32.MaxValue;

            Byte[] data = Encoding.GetEncoding(1251).GetBytes(ReadyQuery);

            stream = client.GetStream();

            // send data
            stream.Write(data, 0, data.Length);

            var sizeBuffer = new byte[10];

            stream.Read(sizeBuffer, 0, 10);
            var sizeMessage = int.Parse(Encoding.GetEncoding(1251).GetString(sizeBuffer, 0, 10));

            data = new Byte[sizeMessage];

            var readSoFar = 0;
            //read data
            while (readSoFar < sizeMessage)
            {
                readSoFar += stream.Read(data, readSoFar, data.Length - readSoFar);
            }

            responseData = Encoding.GetEncoding(1251).GetString(data, 0, data.Length);
            responseData = responseData.TrimStart('&');
            //stream.Close();
            //client.Close();
            //return responseData;
        }
        catch (ArgumentNullException e)
        {
            //return responseData = string.Format("ArgumentNullException: {0}", e);
        }
        catch (SocketException e)
        {
            //return responseData = string.Format("SocketException: {0}", e);
        }
        finally
        {
            if(stream!=null) stream.Close();
            if(client!=null) client.Close();
        }

        return responseData;
    }

And another small suggestion: I think is misleading to have a method named Connect that in fact does more than connect. It will be better to break the Connect method into smaller methods, each one with specific actions (even if they are private).


Fetching records, optionally including deleted records

Lots of ways to approach this one.  Here is how I would approach this (and why).

When working through decisions like this, I tend to think about several concerns:

Maintainability
Readability
Testability
Scalability

Maintainability

A core principal in S.O.L.I.D. programming is the idea of the "O" or the Open closed principal which states:

  Code should be open for extension and closed for modification.

In other words, I shouldn't have to change existing code to accomodate new functionality in an ideal world.

There are several reasons for this:

In some schools of thought, EVERY change is a breaking change (change == bad).
When changing existing code, all dependent code needs to be tested for regressions / compatibility and any derived code must be checked for consistency.
Classes should only change as a result of fixing bugs.

Readability

This is where opinions tend to vary widely, but IMO overloads are much easier to read.

When you use an overload, it allows you to specify an additional set of documentation:

/// <summary>
/// ...description stuff... This method differs from overloads because of x.
/// </summary>

I find this DRAMATICALLY helps readability (and maintainability) for future devs.  It also speaks to the initial intent of your authorship so the functions function is explicit and unlikely to be misunderstood.

Your duplication is also unnecessary since in your case the calls could be chained together:

public IQueryable<ED010_GUIDE> GetGuideRecords(int guideId, int versionId, bool includeDeleted)
{
    var rec = this.GetGuideRecords(guideId, versionId);

    if (!includeDeleted) {
        rec = from x in rec
              where x.DATE_DELETED_DTE == null
              select x;
    }

    return rec;
}

public IQueryable<ED010_GUIDE> GetGuideRecords(int guideId, int versionId)
{
    return from x in ErgDb.ED010_GUIDE
           where x.ERG_EDITION_ID == versionId && x.GUIDE_ID == guideId
           select x;
}

Testability

This one can really go either way in your case.  I would opt for the separation into separate methods most likely because it means that the setup logic for any unit tests would be much simpler as I don't have to account for this or other future scenarios breaking my tests.

Which leads me to my last thought.

Scalability

Or perhaps Flexibility is a better term.

The optional parameters way gets REALLY messy if additional circumstances continue to evolve in the future as you are setting a precedent of non-explicit method signatures.  Future DEVs or indeed yourself are likely to continue the pattern of fragile conditionals, broken unit tests and vague signature usage that can lead to bugs.

Recommendation

Ultimately, I would probably do two things:

Don't do either
Use a "unit-of-work" as an argument

I would go for something like this:

public virtual IQueryable<ED010_GUIDE> GetGuideRecords(GuideQueryRequest request)
{
    var rec = from x in ErgDb.ED010_GUIDE
              where x.ERG_EDITION_ID == request.VersionId && x.GUIDE_ID == request.GuideId
              select x;

    if (!request.IncludeDeletedRecords)
    {
        rec = from x in rec
              where x.DATE_DELETED_DTE == null
              select x;
    }

    return rec;
}

public class GuideQueryRequest
{
    public GuideQueryRequest()
    {
    }

    public int VersionId { get; set; }
    public int GuideId { get; set; }
    public bool IncludeDeletedRecords { get; set; }
}

The inclusion of the GuideQueryRequest type means that you can add new properties to your signature without having to change your signature and break all of the code that depends on it.

This strategy leverages the best aspects of maintainability, flexibility, and readability because you don't need to create additional overloads to supply new arguments, and there are no optional arguments to specify and create ambiguity.

It also gives you a canvas to add robust documentation to the GuideQueryRequest type on how it should be used and what the properties are used for.

In theory, whichever type defines the GetGuideRecord would mark that method as VIRTUAL so that derived types can override the functionality (if new properties are added to the GuideQueryRequest type) and not break the existing functionality.

EDIT

Based on all of the great feedback from @Heslacher, I have updated the above code examples to be correct.

I have also changed the signature of the methods to return IQueryable instances.  It depends on the needs of your application, but I like to have the option of performing additional sorting operations or filtering on a query before the query has been executed.  It also makes more sense given that the method doesn't know what the criteria of the GuideQueryRequest properties will be, it is impossible for this function to know whether or not multiple records (or indeed any records) will be returned.  This delegates the responsibility of how to use the query and the query results to the calling code.

I switched to Linq-to-SQL syntax because I like that format when querying a database and expression syntax when working in memory with results.

For more information on S.O.L.I.D. and a definition of letters and theory, check out these links:

http://www.blackwasp.co.uk/SOLIDPrinciples.aspx
http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod


It is not immediately obvious that getPossibleProducts2() finds all
factorizations  and getPossibleProducts() removes the duplicates,
so the names could be chosen better. (And I would use "factorizations"
instead of "possibleProducts").

Your code is ineffective for two reasons:

The first method finds all factorizations with all possible permutations
(e.g. \$ 120 = 2 \cdot 60 = 60 \cdot 2 \$) so that you have to find the duplicates
in a second step.

As an example, for \$ n = 1024 \$, the first method computes
364176 factorizations, which are then reduced to 627 unique
factorizations in the second step.
You use strings to represents the factorizations (e.g. "2x60", "60x2") which then have to be split into integer arrays again.

To find duplicate factorizations, you split the string (e.g. "60x2")
to an integer array ["60", "2"], sort the factors (["2", "60"]) then concatenate the factors again ("260"). This is used as a hash key
to eliminate the duplicates.
This is quite computing intensive. In particular the hash key

Arrays.stream(y).reduce("", (a, b) -> a + b))

is compute twice, and StringBuilder might be a more effective way
to concatenate the array elements.

More important, it does not work: For \$ n = 130 \$,
the factorizations \$ 2 \cdot 65 \$ and  \$ 5 \cdot 26 \$ are both
mapped to the same hash key "265", so that the output of your
program is

[2, 65]
[13, 2, 5]
[10, 13]
[130]

without the factorization [5, 26].

Generally a better method would be to determine the factorizations
without duplicates in the first step. This can be done recursively
as follows:

Find the smallest factor f of the given number n.
Compute all factorizations of the n / f where the smallest
factor is at least f.

This automatically leads to factorizations with the factors in
increasing order and without duplicates.

As a side effect, the isPrime() function is not needed.

Here is a possible implementation. (Please take this as a demonstration of the algorithm
only. Java is not my first language, so there may be more idiomatic
ways to do implement this.)

public static List<int[]> factorizations(int n) {
    // Start with a minimum factor of 2:
    return factorizations(n, 2, new ArrayList<Integer>());
}

public static List<int[]> factorizations(int n, int minFactor, ArrayList<Integer> previousFactors) {
    List<int[]> result = new ArrayList<int[]>();

    if (n == 1) {
        // This is where the recursion terminates. Convert the list of
        // all previously found factors to an int[] array
        // (found here: http://stackoverflow.com/a/23945015/1187415).
        result.add(previousFactors.stream().mapToInt(i->i).toArray());
    } else {
        for (int factor = minFactor; factor <= n; factor++) {
            if (n % factor == 0) {
                // `factor` is a factor of `n`. Append it to (a clone of)
                // the previously found factors ...
                ArrayList<Integer> factors = new ArrayList<Integer>(previousFactors);
                factors.add(factor);

                // ... and (recursively) find all factorizations of `n / factor`
                // with factors greater or equal to `factor`:
                result.addAll( factorizations(n / factor, factor, factors) );
            }
        }
    }
    return result;
}

Example:

for (int[] f : factorizations(130)) {
    System.out.println(Arrays.toString(f));
}

produces the output

[2, 5, 13]
[2, 65]
[5, 26]
[10, 13]
[130]


Send messages to multiple passed recipients or a default recipient

protected function sendMessages (array $recipients = []) {

    // if no recipients are passed get email from current object (default)
    if ( ! $recipients) {
        array_push($recipients, 'email' => $this->author->email);
    }

    foreach ($recipients as $recipient) {
        //some logic.
    }
}

Assuming you can change the default value, an empty array evaluates as false: https://3v4l.org/Onpcv (alternatively if you don’t like using it like that, do a check on the count, or empty etc)


I like your indentation. It's nice to see well-indented VBA code, it needs to be pointed out!

Application.ScreenUpdating = False

That's generally a good idea, but all by itself, it makes a poor UX - and if calculation mode remains xlAutomatic, it might be a missed opportunity to gain some more performance by turning off calculations while you're working on the worksheet.

Thing is, that's a whole concern all by itself, and as such it deserves its own function. Almost all my macro-enabled Excel workbooks have this function somewhere:

Private Sub ToggleWaitMode(Optional ByVal wait As Boolean = True)
    With Excel.Application

        .Calculation = IIf(wait, xlCalculationManual, xlCalculationAutomatic)
        .Cursor = IIf(wait, xlWait, xlDefault)
        .StatusBar = IIf(wait, "Please wait...", False)

        .DisplayAlerts = Not wait
        .ScreenUpdating = Not wait

    End With

End Sub

So instead of Application.ScreenUpdating = False, you can say ToggleWaitMode, and then do ToggleWaitMode False when you're done processing.

The problem with turning ScreenUpdating off, is that if anything goes wrong, it stays off and the user (and sometimes even the dev) is left thinking Excel is frozen: whenever you play with ScreenUpdating, you must handle runtime errors:

Private Sub DoSomething()
    On Error GoTo CleanFail

    'do something

    CleanExit:
        'clean up code
        ToggleWaitMode False
        Exit Sub
    CleanFail:
        'error-handling code
        Resume CleanExit
End Sub

This way you're always sure that all exit paths involve resetting ScreenUpdating and calculation mode.

...which is a good idea, because the next line can actually blow up whenever a bad file_name1 is passed to the procedure:

Set wbReport = Workbooks.Open(file_name1)

Set wsReport = wbReport.Sheets(1)

I like that: by assigning an object reference, you can work against that object. There are a few issues:

Where's the declaration? Always use Option Explicit and declare all variables! If the variable is declared at module level, then the declaration belongs inside the procedure's scope - move it there, to the smallest possible scope.
The Sheets collection contains chart sheets and actual worksheets. You probably intend to query the Worksheets collection here; if you had declared the wsReport variable As Worksheet and the Sheets(1) object were actually a chart sheet, you'd have a runtime error here.

Select Case wsOutput.Name
    Case Is = "Downtilt Tracker"

Several things here:

You're clearly using the wrong construct here - this should definitely be an If...Then block; what's wrong with this?

If wsOutput.Name = "Downtilt Tracker" Then
    '...
End If

I'm actually surprised Case Is = "string literal" actually compiles and works as intended; it's a pretty convoluted way of doing Case "string literal"...

And then things get a bit out of hand:

        wsReport.Activate
        With Rows(1)
            Set d = .Find("Completed Date")
            If d Is Nothing Then
                MsgBox "Invalid Down-Tilt Tracker." & vbNewLine & "Please import a valid Down-Tilt Tracker."
                wbReport.Close False
                Sheets("Control").Activate
                Application.ScreenUpdating = True
                Exit Sub
            End If
        End With

You have a reference to the wsReport object - you don't need to .Activate it.
The With block adds unnecessary indentation and confusion. I'd do wsReport.Rows(1).Find instead.
d is a meaningless name that doesn't tell anything about what you're doing here.
You need a reference to the "Control" sheet. Declare and assign another Worksheet variable, and use it instead of Selecting and Activateing and using implicit-context Cells.
If d isn't Nothing, you don't appear to be setting ScreenUpdating back to True. The control flow described in my introduction would solve that.

Avoid this - AT ALL COSTS:

wsReport.Activate: wsReport.Cells.Copy: wsOutput.Activate: Cells(1, 1).Select: ActiveSheet.Paste: Cells.EntireColumn.AutoFit: wsOutput.Range("A1:AB" & 1048576).HorizontalAlignment = xlCenter

There are 7 instructions on that line of code; there's no reason to do this. Ever.

And here's where I'd think your bottleneck is:

wsReport.Cells.Copy

You're copying the entire worksheet.

wsOutput.Range("A1:AB" & 1048576)

Why bother with the concatenation here? It's hard-coded anyway, and besides, A1 has its row specified inside the string literal.

wsOutput.Range("A1:AB1048576")

This might work a bit better - if not, it's at least certainly much easier to read:

wsReport.UsedRange.Copy
wsOutput.Range("A1").Paste
wsOutput.UsedRange.Columns.EntireColumn.AutoFit
wsOutput.UsedRange.HoriontalAlignment = xlCenter


Program to determine ranges of char, short, int and long variables, both signed and unsigned

Signedness of unqualified char is implementation defined. It may well be possible that char is in fact unsigned. Change char to signed char.
A char is not guaranteed to have 8 bits (it is guaranteed to have at least 8 bits). Use CHAR_BIT instead.
Narrowing types (e.g. assigning long to char) always make me uncomfortable. A better technique to get a value with only MSB set is (TYPE is either char,int,long,whatever)

unsigned TYPE value = ~(((unsigned TYPE) -1) >> 1)


Scraping the names of a whole category with a three-liner code

I would add more lines, but improve on performance:

parse only the relevant parts of the page with SoupStrainer
re-use the same session when making requests

Improved version:

import requests
from bs4 import BeautifulSoup, SoupStrainer

URL_TEMPLATE = "https://www.yellowpages.com/search?search_terms=coffee&geo_location_terms=Los+Angeles%2C+CA&page={0}"

parse_only = SoupStrainer("h2", class_="n")

with requests.Session() as session:
    for page_number in range(1, 52):
        response = session.get(URL_TEMPLATE.format(page_number))

        soup = BeautifulSoup(response.content, "lxml", parse_only=parse_only)
        for title in soup.select("h2.n"):
            print(title.get_text())

Also note a variable name change - page_number would be more descriptive than i.


Fluent NHibernate, Getting or Creating a new `Tag`

Admittedly I am using a really old version of nhibernate, but I think this is still there (to replace a few lines):

Restrictions.InG("Name", tagNames.ToList())

I think I would also bring the using statement outside of the loop:

private IEnumerable<Tag> GetOrCreateTags(IEnumerable<string> tagNames)
    {
        var tagList = tagNames.ToList(); //or make the parameter above an ICollection<string>

        var tagCriteria = _nHibernateSession.CreateCriteria<Tag>();
        tagCriteria.Add(Restrictions.InG("Name", tagList));

        var tagsFromTagNames = tagCriteria.List<Tag>();

        //naming this for readability
        var existingTagNames = tagsFromTagNames.Select(x => x.Name);

        var newTags = tagList
            .Except(existingTagNames)
            .Select(t => new Tag
                           {
                               Name = newTagNeverSeen,
                               Enabled = true,
                               DateCreated = DateTime.Now
                           }); //moving this outside the loop makes the loop cleaner

        if (newTags.Any())
        {
            using (var transaction = _nHibernateSession.BeginTransaction())
            {
                foreach (var tag in newTags)
                {
                    _nHibernateSession.SaveOrUpdate(tag);
                    tagsFromTagNames.Add(tag);
                }
                transaction.Commit();
            }
        }
        return tagsFromTagNames;
    }


Showing a spinner over content - fade out some elements and disable clicking on some elements

First, For the universal selector, the performance diference will be negligible exept if you load 100s or 1000s of those element with into one web page.

This said, I reviewed you code and made several changes :

Used pseudo elements to make the spinner and the overlay for 2 reasons :

one unsemantic html tag removed
this alows to play with z-index so the red elements can be behind or in front of the overlay.

Used z-index instead of opacity to fade red elements out and moved the pointer-events property to the overlay pseudo element so that the .subContent can still be clicked.

.content {
    width: 180px; height: 180px;
    position: relative;
    padding: 10px;
}
.is-showingSpinner:before, .is-showingSpinner:after{
    content:'';
    position:absolute;
    bottom: 0; left: 0; right: 0; top: 0;
    z-index:2;
    pointer-events:none;
}
.is-showingSpinner:before {
    width:100%; height:100%;
    background-color: rgba(211, 211, 211, .8);
}
.is-showingSpinner:after {
    margin:auto;
    width: 32px; height: 32px;
    border: 4px solid #4ea6ea;
    border-left-color: transparent;
    border-top-color: transparent;
    border-radius: 50%;
    -webkit-animation: spin 600ms infinite linear;
    animation: spin 600ms infinite linear;
}
.content-subContent {
    width: 40px; height: 40px;
    background-color: red;
    display: inline-block;
}
@-webkit-keyframes spin {
    0%   { transform: rotate(0deg); }
    100% { transform: rotate(359deg); }
}
/* TODO: This seems like a poor way of doing this. What would be more robust? */
 .is-showingSpinner > .is-aboveFadeout {
    position:relative;
    z-index:2;
}
.is-alwaysClickable:hover {
    cursor: pointer;
    background-color: #8b0000;
}

I also removed the classes you added to the spinner element. If you still need them, you can add them to the .content div to style the spinner ( :after pseudo element) as needed.


A one liner for your problem:

def slice_and_multiply(arr):
    return np.product(arr, axis=1, keepdims=True) / M

This multiplies all of the "columns" first, then divides by each "column" to get the result of multiplying all but that column, which may be problematic depending on your data due to potential overflows or precision losses. And it breaks completely if there are any zeros in the input array.

If you can get rid of the division (this is a typical interview question, by the way), then all of those problems dissappear. And you can do it with relative ease, by cleverly storing accumulated products from both ends of the array. then multiplying them together:

def slice_and_multiply_bis(arr):
    ret = np.ones_like(arr)
    np.cumprod(arr[:, :-1], axis=1, out=ret[:, 1:])
    ret[:, :-1] *= np.cumprod(arr[:, :0:-1], axis=1)[:, ::-1]
    return ret

And of course:

>>> M = np.random.rand(16, 8, 1000, 255)
>>> np.allclose(slice_and_multiply(M), slice_and_multiply_bis(M))
True

That last code is a little too clever for its own good, so it should it should be heavily commented.

By my timings, for your target size, slice_and_multiply is 5x faster than your implementation, and slice_and_multiply_bis about 2-3x faster.


In ImageToBase64String() you do not dispose your stream. GC will do it but server-side is never never a good idea to postpone resource deallocation:

static string ImageToBase64String(Bitmap img)
{
    using (var ms = new System.IO.MemoryStream())
    {
        img.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
        return Convert.ToBase64String(ms.ToArray());
    }
}

Note that if you use this feature often you may reduce memory pressure reusing same buffer, you may implement an ObjectPool<MemoryStream> class.

You create a new bitmap just for resizing then yes, it has to be disposed too:

private static string ResizeAndConvertToBase64(Stream stream)
{
    using (var image = new Bitmap(FileUploadControl.FileContent))
    {
        if (IsTooBig(image))
        {
            using (var scaledImage = ScaleDownImage(image))
            {
                return ImageToBase64String(scaledImage);
            }
        }

        return ImageToBase64String(image);
    }
}

private static bool IsTooBig(Bitmap bitmap)
{
    return bitmap.Width > MaximumWidth || bitmap.Height > MaximumHeight;
}

Note I also moved magic numbers from code to constants MaximumWidth and MaximumHeight. Now you still have to deal with ScaleDownImage(). First of all you may want to keep image aspect ratio then I'd first move this calculation to another separate function (I dropped settings unused for image drawing):

private static Bitmap ScaleDownImage(Bitmap image)
{
    var newSize = ResolveScaledDownImageSize(image);
    var scaled = new Bitmap(newSize.Width, newSize.Height, PixelFormat.Format32bppArgb);

    using (var g = Graphics.FromImage(scaled))
    {
        g.CompositingQuality = CompositingQuality.HighQuality;
        g.InterpolationMode = InterpolationMode.HighQualityBicubic;
        g.SmoothingMode = SmoothingMode.AntiAlias;
        g.PixelOffsetMode = PixelOffsetMode.HighQuality;

        g.DrawImage(image, new Rectangle(Point.Empty, newSize));
    }

    return scaled;
}

There is also a built-in method to re-scale images, you should check if it satisfy your needing, in this case drop ScaleDownImage() and change ResizeAndConvertToBase64() as follow:

private static string ResizeAndConvertToBase64(Stream stream)
{
    using (var image = new Bitmap(FileUploadControl.FileContent))
    {
        if (IsTooBig(image))
        {
            using (var scaledImage = new Bitmap(image, ResolveScaledDownImageSize(image)))
            {
                return ImageToBase64String(scaledImage);
            }
        }

        return ImageToBase64String(image);
    }
}

In general don't forget to always call Dispose() for classes which implement IDisposable and whenever possible use using statement to simplify that code.

Now only ResolveScaledDownImageSize() function is empty. If you do not need to keep aspect ratio you may simply:

private static ResolveScaledDownImageSize(Bitmap image)
{
    return new Size(
        Math.Min(MaximumWidth, image.Width),
        Math.Min(MaximumHeight, image.Height));
}

To keep aspect ratio you may keep width or height (according to which one is bigger than required), you can use one of these to perform calculation:

\$height_{new} = \frac{height}{width} width_{new}\$

or

\$width_{new} = \frac{width}{height} height_{new}\$


Yes, it is unnecessary to create a sequence, then add it. Just simply add while counting up. Take a look at my question and this answer; it will simply create the sequence while adding, without even having to store the sequence, just the result and the previous numbers. It takes the advantage of the fact that every three Fibonacci numbers in the sequence is an even number, so inside the while loop, all it needs to do, is add the result, then skip 2 numbers. The code (for convenience) is down below:

public class EvenFibonacciFinder {
    private static final int MAX_NUM = 4_000_000;

    public static void main(String[] args)
    {
        long time = System.nanoTime();
        int fib1 = 1;
        int fib2 = 2;
        long sum = 0;

        while (f2 <= MAX_NUM) {
            int fib3;
            sum += fib2;
            // This skips three ahead in the sequence.
            fib3 = fib1 + fib2;
            fib1 = fib2 + fib3;
            fib2 = fib1 + fib3;
        }
        long end = System.nanoTime();
        System.out.println("Result: " + sum +
                "\nTime used for calculation in nanoseconds: " +
                (end - time));
    }
}


LeetCode 894: All Possible Full Binary Trees

Make clone_tree() private
Make helper functions that are not part of the public API private.
Avoid unnecessary nesting of statements if possible
In allPossibleFBT() there's a lot of indentation. There's a risk of the code running off the right hand side of the screen, making it hard to read. Try to reduce nesting if possible (but only if it improves readability and maintainability). In this case, you can get rid of the outermost if-statements by doing an early return if n is even, and then unconditionally adding the single-node tree:
static std::vector<TreeNode *> allPossibleFBT(const size_t n) {
    // Exit early if n is even
    if ((n & 1) == 0)
        return {};

    // Start with the one-node tree
    std::vector<TreeNode *> full_binary_trees{new TreeNode(0)};

    // Add more complex trees
    for (size_t index = 2; ...) {
        ...
    }

    return full_binary_trees;
}

Take a reference of an object if you are using it a lot
When we see a common expression being reused a lot, we can create a new variable holding the result of that expression. When the expression is actually an object, and we don't want to copy the object, we can still avoid repeating the expression by creating a reference to that object. So instead of explicitly writing full_binary_trees.back() many times, write:
auto &root = full_binary_trees.emplace_back(new TreeNode(0));

if (right_index == right_size() - 1) {
    root->left = ...

Since C++17, emplace_back() returns a reference to the emplaced element. In case you are writing code for older standards, you could write:
full_binary_trees.emplace_back(new TreeNode(0));
auto &root = full_binary_trees.back();
...

But perhaps it would even be better to first created the new tree, and then append it to the vector at the end:
auto root = new TreeNode(0);

if ...

full_binary_trees.emplace_back(root);

Other possible improvements
It would be nice to get rid of the inner-most if-statements, which is possible if you don't care about memory leaks:
for (auto left: allPossibleFBT(index - 1)) {
    for (auto right: allPossibleFBT(n - index)) {
        auto root = new TreeNode(0);
        root->left = clone_tree(left);
        root->right = clone_tree(right);
        full_binary_trees.emplace_back(root);
    }
}

In the context of this LeetCode problem, which doesn't state whether or not you are allowed to link a node into multiple trees, you might even get away with not calling clone_tree() at all, but just writing root->left = left; root->right = right.
Also note that this problem might benefit from memoization.


Social network broadcast message algorithm

1. Bug

The algorithm does not always find the minimum number of messages! Suppose that we have the following follower graph (an arrow from A to B means that A follows B).

To reach everyone, only one message is needed (to C or D; either will do). But when I represent this graph as an adjacency matrix:

connectMatrix = [
    [1, 1, 1, 0],
    [1, 1, 0, 0],
    [0, 0, 1, 1],
    [0, 0, 1, 1],
]

and run the code from the post, I get the following output:

[0, 1]
[3, 2]
persons to send message,  [0, 3]

2. Review

The code is not portable to Python 3 because it uses the print statement. It would be easy to make it portable by using from __future__ import print_function.
The code is awkward to test because it only runs as a script and so it can't be run from the interactive interpreter, or from a unit test case. It would be better to organize the code into functions. For example, I would write:

def min_sinks(connectMatrix):
    """Return the smallest list of sinks such that each node in the graph
    has a path to at least one of the sinks.

    """
    inDegree = [0] * len(connectMatrix)
    visited = [0] * len(connectMatrix)
    allVisited = 0
    # ... etc ...

But this doesn't quite work because BFS uses the global variable allVisited. The way to fix that is to observe that BFS adds one to allVisited for each node it adds to result, and so in min_sinks we can update allVisited as follows:

visited_nodes = BFS(connectMatrix, visited, nextIndex)
allVisited += len(visited_nodes)

The name allVisited doesn't describe the variable very well: better would be something like visited_count.
The code in BFS uses the name persons for the nodes in the graph. But nothing in this code depends on the fact that the nodes represent persons. So it would be better to use a neutral name like nodes.
Popping the leftmost item of a list takes time proportional to the length of the list (see the time complexity page on the Python wiki). It's necessary to use a collections.deque to ensure that popping is efficient.
The data structure visitedNodes is a list whose value at index i is 1 if the node i has been visited, and 0 if it has not. It would be clearer to use True and False instead of 1 and 0.
The code in the post uses an adjacency matrix to represent the graph. But this is a poor representation for this algorithm, for two reasons. First, an adjacency matrix takes \$Θ(n^2)\$ space to store a graph on \$n\$ nodes, and this is only efficient if the graph is dense, whereas a graph of Twitter followers is (most likely) sparse. Second, at each iteration of the breadth-first search, the code needs to iterate over the followers of a node, and that takes \$Θ(n)\$ in the adjacency matrix represention, which is inefficient if the graph is spare.

It would be better to use a different representation, for example adjacency list.

3. Correct algorithm

The correct algorithm for finding the minimum number of sinks so that every node has a path to at least one sink is as follows:

Find the strongly connected components in the graph, for example using Tarjan's algorithm. (A strongly connected component is a group of nodes such that there is a path from every node in the component to every other node in the component. For example, a cycle.)
In the graph, replace each strongly connected component with a new single node (updating the edges so that if there was an edge between a node \$n\$ and any node in the component, there is a corresponding edge between \$n\$ and the new node).
If the graph has no nodes, stop.
Find a node of out-degree zero and add it to the set of sinks. (If the node replaced a strongly connected component in step 2, then add any node from the original component.)
Remove the sink from the graph, along with every node that has a path to the sink.
Go to step 3.

For example, suppose that we start with this graph:

At step 1 we find the strongly connected components:

At step 2 we replace each component with a new single node:

At step 4 we find that there is one node of out-degree zero (the node CD) so we add either C or D to the set of sinks (it doesn't matter which). At step 5 we remove CD from the graph together with every node that has a path to CD. This leaves the graph empty, so we are done.

4. Answers to questions

You asked in comments whether "out-degree zero" in step 4 of the algorithm is a mistake for "in-degree zero".

The answer is no: "out-degree" is correct. Remember that the graph is a follower graph. So a node with out-degree zero represents someone who doesn't follow anyone (or a strongly connected component nones of whose members follows anyone outside of the component). The only way that someone who doesn't follow anyone can receive the message is if they receive it directly. So that person needs to be added to the set of sinks.
You asked for clarification about the direction of edges in the graph. I took this directly from the problem statement:

  Also, we need to consider loops like, if A follows B, B follows C, C follows D, D follows A (A → B → C → D → A) then reaching only one of them is sufficient to spread your message.

This says that "A follows B" implies "A → B" which implies that there is an edge from A to B. I stated this explicitly in §1 above, where I wrote:

  (an arrow from A to B means that A follows B)

Of course you are free to represent the graph the other way round, so that if A follows B then B → A. If you do this then this doesn't affect the strongly connected component part of the algorithm (since a component remains strongly connected if you reverse all the edges), but you will need to swap "out-neighbour" and "in-neighbour" in the remainder of the algorithm.


The general approach seems reasonable to me, thought I believe it means you have to very careful around anything that can switch threads, like TPL or async-await.

public static ContextIdentity LocalIdentity;

Why is this public? From your description, it seems to me that users should never read this.

If this is so that users can set this, maybe a setter on Current would make more sense.

Also, you should use properties instead of public fields.

return Thread.CurrentPrincipal.Identity as ContextIdentity;

You already know that Indentity is ContextIdentity, so you should use a cast, not as. That way, if you make a mistake, you're going to get an exception, not null.

_name = "Default";

Why is _name a field, not a property like the others? If you don't want users changing the name, you can make the setter private.

"True".Equals(ConfigurationManager.AppSettings["DatabaseIntegratedSecurity"])

This is not Java, you can use == "True".


def get_range_list(amount, val, end):
    return [
        ('flcon1:DataValue~Osiris_Test_Data_' + str(i + 1) + end, randint(0, val))
        for i in range(amount)
    ]

You're only interested in the value of amount, not in the variable it produces itself. Have you tried using xrange instead? xrange is much faster than range.

Afar from that, I cut-out the OPC parts and ran the rest through a profiler.

python -m cProfile -s tottime CR_135184.py 1000000 1

Cut code:

import argparse
import sys
import time
from random import randint

def get_args():
    parser = argparse.ArgumentParser(description='Record data value changes')
    parser.add_argument('range', type=int, help='range of values to record')
    parser.add_argument('time', type=int, help='time to update values in seconds')
    return parser.parse_args()

def get_range_list(amount, val, end):
    return [
        ('flcon1:DataValue~Osiris_Test_Data_' + str(i + 1) + end, randint(0, val))
        for i in xrange(amount)
    ]

def get_time_list(amount, end):
    return [
        ('flcon1:DataValue~Osiris_Test_Data_' + str(i + 1) + end, time.strftime('%m/%d/%Y %H:%M:%S'))
        for i in xrange(amount)
    ]

def main():
    args = get_args()
    print('Logging test harness value changes')

    t0 = time.clock()
    value_list = get_range_list(args.range, 9, '.Value')
    time_list = get_time_list(args.range, '.TimeStamp')

    print(value_list)
    print(time_list)

    print('Wrote values')

    with open(str(args.range) + '.' + str(args.time) + '_1-iccp_harness.log', 'a') as f:
        for i in xrange(0, args.range):
            f.write('Osiris_Test_Data_' + str(i + 1) + ', realQTimetagExtended' +
                    ', %g, %s, %s\n' % (value_list[i][1], 00, time_list[i][1]))

    t1 = time.clock()
    print t1-t0

    print('Logged values')

if __name__ == '__main__':
    main()

Result (the relevant parts):

Wrote values
7.145332
Logged values
         5001665 function calls (5001624 primitive calls) in 15.432 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1   11.660   11.660   15.299   15.299 CR_135184.py:27(main)
  1000000    1.269    0.000    1.269    0.000 {time.strftime}
  1000000    0.876    0.000    0.925    0.000 random.py:175(randrange)
        1    0.551    0.551    1.708    1.708 CR_135184.py:13(get_range_list)
        1    0.510    0.510    1.779    1.779 CR_135184.py:20(get_time_list)
  1000000    0.232    0.000    1.157    0.000 random.py:238(randint)
  1000000    0.151    0.000    0.151    0.000 {method 'write' of 'file' objects}
        1    0.120    0.120   15.432   15.432 CR_135184.py:1(<module>)
  1000000    0.049    0.000    0.049    0.000 {method 'random' of '_random.Random' objects}

Basically, you're slowing down your code by checking the time. The generating of random numbers takes some time, but get_range_list and get_time_list are surprisingly fast.

The real bottleneck? Your OPC connection. And I'm not sure whether this can be fixed at all in code.


Is this file I/O utility efficient or is there a lot of fat?

DataBase is a poor name for a collection of file I/O convenience routines.  Especially one whose readFile() function swallows empty lines and strips
 tags as it reads.  What exactly are you trying to accomplish with this class?  I suspect that you have decomposed your problem poorly, and this class shouldn't exist.

The getExtension() function could be more succinct.

public static String getExtension(File f) {
    String s = f.getName();
    int i = s.lastIndexOf('.');

    return (i > 1 && i < s.length() - 1) ?
        s.substring(i + 1).toLowerCase() :
        null;
}

I've changed the condition from i > 0 to i > 1 since the names of Unix dotfiles (e.g. .profile) are not usually thought of as file extensions.  That's a judgement call for you to make.

I think your readFile() should take a File rather than a String parameter.  If you design your API so that you consistently use File to represent file paths and String to represent file content data, then it will be harder to accidentally call your function with swapped arguments (especially for writeToFile()).  Also, if you call the other function writeToFile(), then you should rename readFile() to readFromFile() for consistency.

Swallowing exceptions is usually a bad idea, even if you print the stack trace.  I would just let any IOExceptions propagate.

You're just using the default character encoding, so you don't need to bother with constructing InputStreams — just use Readers.

Testing for String.contains() before doing String.replaceAll() just repeats the work needlessly.  String.replaceAll() will just return the original string anyway if no replacement needs to be performed.

I don't know what you are trying to accomplish with the tempLine stuff.  Since you initialize tempLine = "", it will always be true that tempLine.trim().length() is 0, so all of that is dead code.

Assuming you don't need thread safety, just use StringBuilder instead of StringBuffer.

Doing a string concatenation in buff.append(line + "\n") defeats the efficiency gains of using a StringBuffer.

All together…

public static String readFromFile(File f) throws IOException {
    if (!f.exists()) {
        // I find this behavior surprising; I would have expected a
        // FileNotFoundException to be thrown instead.
        return "";
    }

    FileReader r = new FileReader(f);
    BufferedReader br = null;
    try {
        br = new BufferedReader(r);
        StringBuilder buff = new StringBuilder();

        String line;
        while (null != (line = br.readLine())) {
            if (line.length() > 0) {
                line = line.replaceAll("
", " ");
                buff.append(line).append('\n');
            }
        }
        return buff.toString();
    } finally {
        if (br != null) br.close();
        r.close();
    }
}

The code could be a bit less clumsy if you use Java 7's try-with blocks.

writeToFile() can be simplified by a lot.  For simplicity, I would just use a FileWriter instead of the java.nio API.

public static void writeToFile(File f, String data, boolean append) throws IOException {

    File parent = f.getParentFile();
    if (!parent.exists()) {
        parent.mkdirs();
    }

    FileWriter w = new FileWriter(f, append);
    try {
        w.write(data);
    } finally {
        w.close();
    }
}

With the simplification to writeToFile(), getPathSeparator() might not be necessary anymore.


Slick Slider – change contents based on current slide data-index

I would combine the two lines where you get the dataIndex, rename that to dataId to be consistent with the HTML element, and replace the loop by hiding everything, and then showing the correct element by finding it with a jQuery selector:

$('.slider').on('afterChange', function(event, slick, currentSlide, nextSlide){
    var dataId = $(slick.$slides[currentSlide]).data('index');
    $('.content').hide();
    $('.content[data-id=' + dataId + ']').show();
});

I also noticed that looking up dataId is superfluous, since it is simply the value of currentSlide plus one.  So you could also make this change and retain the same behavior:

$('.slider').on('afterChange', function(event, slick, currentSlide, nextSlide){
  $('.content').hide();
  $('.content[data-id=' + (currentSlide + 1) + ']').show();
});


There are formatting and whitespace issues in your code, fix them! You're also switching between declaring variables and initializing them in the next line and initializing them in the same line, be consistent!

if(i<1){

That hurts! What is i? Where does it come from? Why do you check against less then one? Keep your variable names always clear, f.e. numberOfLinesInFile is way better then i in my opinion.

new File(newDir.toString() + "\\" + name + ".aqua");

This assumes that it is run on a Windows machine (or anything else that uses a backslash as path separator...which I hope no one else does). That will lead lead to problems on *nix machines, but I know that / works on any platform, you could also go full length and use File.separator.

Or you use the constructor-overload that is provided for exactly these means:

new File(newDir.toString(), name + ".aqua"):

String out = "";

That's not a great name for that variable, text or content would be better suited.

out+=line + "\n";

This has one problem, concatenating strings with the + is slow like whatever-place-of-ternal-pain-you-believe-in-if-any. Use a StringBuilder, that will speed things up considerable.

jTextArea1.setText(out);

Again, not a good name.


Passing Node objects into insert routine of linked list

You could delegate the construction of nodes to the insert function :

#!/usr/bin/python

# linked list in Python

class Node:
    """linked list node"""
    def __init__(self, data):
        self.next = None
        self.data = data
    def insert(self, data):
        n = Node(data)
        n.next = self.next
        self.next = n
        return n
    def showall(self):
        print self.data
        if self.next != None:
            self.next.showall()

ll = Node("one")
ll.insert("two").insert("three")
ll.showall()

You could make this more concise by adding a default argument to your __init__() :

class Node:
    """linked list node"""
    def __init__(self, data, next=None):
        self.next = next
        self.data = data
    def insert(self, data):
        self.next = Node(data, self.next)
        return self.next
    def showall(self):
        print self.data
        if self.next != None:
            self.next.showall()

ll = Node("one")
ll.insert("two").insert("three")
ll.showall()

Additional point : from PEP 8 :

  Comparisons to singletons like None should always be done with is or
  is not, never the equality operators.


Extracting text fields from  tags in an HTML message

I know this is Code Review not Rewrite My Code, however I would suggest using a third-party Html parser (like the Html Agility Pack for example) over regular expressions if that's an option.

I realize you're doing very trivial parsing here, but from my personal experiences regular expressions grow to unmaintainable status quicker than anything in software development.

If you were to use a Html parser, you could do something like this:

string htmlToParse = "
Some text this is a test

Some more text this is another test
";
const string ElementToParse = "span";
const string IdField = "FieldId";

HtmlDocument htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(htmlToParse);

int fieldId = default( int );

Dictionary<int,string> fieldValuesTable =
(
    from
        htmlNode in htmlDocument.DocumentNode.DescendantNodes()
    where
        htmlNode.Name.Equals( ElementToParse, StringComparison.InvariantCultureIgnoreCase )
        &&
        htmlNode.Attributes.Contains( IdField )
    let
        id = htmlNode.Attributes[ IdField ].Value
    where
        Int32.TryParse( id.Substring( id.IndexOf( "-" ) + 1 ), out fieldId ) // this is stil not ideal,
    select
        new { Id = fieldId, Text = htmlNode.InnerText }
).ToDictionary( f => f.Id, f => f.Text );

You get the output:

4 : Some text
5 : Some more text

IMHO, it's much cleaner and maintainable.


Implementation

operator==(SplitStringIterator const&) only compares pointers for equality, not lengths. This might not be intended.
The ForwardIterator concept isn't actually fulfilled: For that, operator* has to return a reference! (And it makes sense, too: You cannot provide a multipass guarantee in case the contents get modified, i.e. a previously different CharT value gets changed to delim.)
Why restrict delim to CharT, when it could just as easily be a std::basic_string_view<CharT>? The advance_() logic would have to be updated for the varying length, but that's the only non-type change I can see that would be necessary. (Though one could add a conversion or a special case for single CharT delimiters.)

Small caveat: If parts_indexable had a longer lifetime than data (e.g. it got returned by foo), all its contents would dangle (though this is inherent problem of std::string_view, and not of your implementation). This could be fixed by making explicit copies by changing the type of parts_indexable to std::vector<std::string>. (Not a criticism of your work, just a note for users of this iterator).


Scraping names of directors from a website

DRY

Reduce duplicated logic using helper functions.
Currently you have 2 lines of code for each field you extract,
for example:

companys = sel.xpath('//*[@id="company-name"]/text()').extract()
companys = [company.strip() for company in companys]

names = sel.xpath('//*[@id="%s"]/a/text()' % string1).extract()
names = [name.strip() for name in names]

This is tedious. You could capture the common logic in a helper function, for example:

def to_list(xpath):
    return [v.strip() for v in xpath.extract()]

With that, much of the code can be simplified:

companys = to_list(sel.xpath('//*[@id="company-name"]/text()'))
names = to_list(sel.xpath('//*[@id="%s"]/a/text()' % string1).extract())

Repeated operations

Here, an xpath lookup is performed twice:

if sel.xpath('//*[@id="%s"]/text()' % string3):
    dateofbirths = sel.xpath('//*[@id="%s"]/text()' % string3).extract()
else:
    dateofbirths = ["n/a"]
dateofbirths = [dateofbirth.strip() for dateofbirth in dateofbirths]

It would be better to avoid that:

dateofbirths = to_list(sel.xpath('//*[@id="%s"]/text()' % string3))
if not dateofbirths:
    dateofbirths = ["n/a"]

Use "...".format(...)

The "%s" % ... style formatting is old, it's recommended to use the format function instead, for example:

names = sel.xpath('//*[@id="{}"]/a/text()'.format(string1)).extract()

Formatting

Python has a style guide called PEP8, I suggest to follow it.


Example-ish code

It's clear that you have anonymized the procedure code by replacing real names by example names. In the future, it is better to use your real object & variable names, as someone may actually have things to say to improve your naming, which is one of the most important aspects to code clarity/readability.

Structure of procedure

There are some really strange things about the way your procedure is structured. For one, the test string that it's meant to parse is hard-coded into the procedure code.

/* This passes as well. */
DECLARE myList varchar(255) default ' ,, ,   1,2  ,3, 456 , 7 8 9 ABC ,,  ';

I feel sure that this is temporary, but if one of my colleague SQL developers handed that to me for a code review, I would tell them they need to go finish the code first so it's actually usable for any test case. So the string to parse should be an argument to the procedure...

CREATE PROCEDURE `sp_ParseCommaSeparatedString`(
  in p_commaSeparatedString varchar(255),
  in p_willBeUsedLater int(10) unsigned
)

And as such, you can just document your test cases in block comment with procedure calls. This also documents to a reasonable extent how the procedure is meant to be used.

/* Test Cases - PASSED
call sp_ParseCommaSeparatedString('');   --0 rows
call sp_ParseCommaSeparatedString(null); --0 rows
call sp_ParseCommaSeparatedString('1');  --1 row
call sp_ParseCommaSeparatedString('1,2');--2 rows
--Complex test case, 5 rows
call sp_ParseCommaSeparatedString(' ,, ,   1,2  ,3, 456 , 7 8 9 ABC ,,  ');
*/

Unused parameter

This is dead code, as the name indicates: in p_willBeUsedLater int(10) unsigned

It would be more beneficial to not add it in until it is ready, so that the stored procedure calls aren't unnecessarily confusing. You would have 2 options here which would be better. MySQL is a bit clunky for something like this, as far as SQL goes, but it can still be done fine.

1) Document as a TODO

CREATE PROCEDURE `sp_ParseCommaSeparatedString`(
  in p_commaSeparatedString varchar(255)
)
  MODIFIES SQL DATA
myProc:begin

/*
 * TODO: implement additional input param:
 *   in p_someSensiblyNamedNumber int(10) unsigned
 * This parameter will be used for [...]
 */

2) Provide a default argument

CREATE PROCEDURE `sp_ParseCommaSeparatedString`(
  in p_commaSeparatedString varchar(255),
  in p_someSensiblyNamedNumber int(10) unsigned
)
  MODIFIES SQL DATA
myProc:begin

IF p_someSensiblyNamedNumber IS NULL THEN
  SET p_someSensiblyNamedNumber = 36;

Either will work fine and which is better really depends on what you are needing to do with it. From the code your presented, that parameter is not used for anything right now and it's not clear from what you named it what it is supposed to do, so... hard to say.

Now we can clean up the top a bit.

DECLARE stringToParse varchar(255);
SET stringToParse = TRIM(p_commaSeparatedString);

Results table

At the moment, you are finding yourself having to delete the rows from the table before you call the procedure, which is not ideal at all and will lead to errors. Instead, do that in your procedure, either at the top when you start, or just before you start inserting values in it (in case the code prior to that takes long, this would mitigate the possibility of another process adding rows into the same table which this is running).

Just check if there are any rows in it, if there are not just skip, and if there are, delete them.

IF EXISTS (SELECT 1 FROM example) THEN
    DELETE FROM example;
END IF;

WHILE (length(myList) > 0)
DO...

Overall approach

SQL clearly was not designed to parse strings of text like a more general language/domain could; if things are not in tables, rows and columns, they are much more difficult to manipulate with SQL. With that in mind, I would say your logic is probably about as good as it is going to get, while still maintaining readability.

Perhaps consider adding a small amount of code comments in the conditionals in your last WHILE section so that the next person who has to look and/or debug this code (which may very well be your future self) will not have too much difficulty.

Nitpicks

DELIMITER ;;

This would make more sense if you set the delimiter to something that looks less like the normal delimiter. I have most often seen used $$, // and || as delimiters.

trim(substring(myList,LOCATE(',',myList)+1,length(myList)));

Try to get in the habit of spacing your operators, commas, etc. to make it easier to read. Also, you are using mixed capitalization of keywords, which just looks a bit messy. Pick one capitalization style and stick to it.

TRIM(SUBSTRING(myList, LOCATE(',', myList) + 1, LENGTH(myList)));


Linq-to-Sql Contains an int? inside a list of int

Another way to do this is to remove NULL values from the comparison to begin with, like so:

return Context.TblNumbers
              .Where(m => m.Number != null)
              .Where(m => convertedNumbers.Contains(m.Number.Value))
              .Select(m => m.Number.Value)
              .ToList();

I think this also reads more naturally since you only have the chance to match on non-null values anyway, instead of GetValueOrDefault() which implies you might need to get a default value.