Longest common substring approach using suffix arrays

OK then: since your concern is speed, let's track our progress with actual timing data. The first step is to run the code through the Python profiler. With the addition of a bit of driver code that just calls LongestCommonSubstr().longest_common_substr 10000 times, I get the following results:

$ python3.4 -m profile lcs_profile.py abcdeffghiklnopqr bdefghijklmnopmnop
         1130008 function calls in 3.083 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 :0(__build_class__)
        1    0.000    0.000    3.082    3.082 :0(exec)
        1    0.000    0.000    0.000    0.000 :0(hasattr)
   280000    0.257    0.000    0.257    0.000 :0(len)
   260000    0.330    0.000    0.330    0.000 :0(max)
   260000    0.350    0.000    0.350    0.000 :0(min)
        1    0.000    0.000    0.000    0.000 :0(setprofile)
    10000    0.042    0.000    0.042    0.000 :0(sorted)
        1    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:2264(_handle_fromlist)
   260000    1.067    0.000    1.746    0.000 genericpath.py:69(commonprefix)
        1    0.022    0.022    3.082    3.082 lcs_profile.py:1(<module>)
    20000    0.069    0.000    0.157    0.000 lcs_profile.py:11(_get_suffix_str)
    20000    0.071    0.000    0.071    0.000 lcs_profile.py:13(<listcomp>)
    10000    0.807    0.000    2.794    0.000 lcs_profile.py:15(_get_lcsubstr)
        1    0.000    0.000    0.000    0.000 lcs_profile.py:3(LongestCommonSubstr)
    10000    0.067    0.000    3.060    0.000 lcs_profile.py:4(__init__)
        1    0.000    0.000    3.083    3.083 profile:0(<code object <module> at 0x10996f030, file "lcs_profile.py", line 1>)
        0    0.000             0.000          profile:0(profiler)

The most important column is the second one, tottime, marking the total amount of time consumed by each function (but not the functions it calls). The largest entries in that column are for the commonprefix function and the _get_lc_substr function, so those are the spots you should focus on optimizing.

To keep track of our progress, I coupled your class with the following driver program which prints the execution time for 10000 runs:

def lcs_longest_match(a, b):
    return LongestCommonSubstr(a, b).longest_common_substr
if __name__ == '__main__':
    import timeit, sys
    print('lcs    ', timeit.timeit('lcs_longest_match(sys.argv[1], sys.argv[2])',
                                   setup='from __main__ import lcs_longest_match',
                                   number=10000))

(this is for Python 3). A first run on my computer gives

$ python3.4 lcs.py abcdeffghiklnopqr bdefghijklmnopmnop
lcs     0.49046793299203273

I'll start with _get_lc_substr, since you wrote that code and it'll be easier to work through. The algorithm you use is to go through each pair of consecutive strings in the sorted suffix array, find the common prefix, and save that prefix only if it's longer than any common prefix already found. I can suggest a few improvements:

Iterating over pairs of consecutive elements is a common task that has a fairly standard recipe, pairwise(iterable), given in the documentation for the itertools module. You can use the implementation from the more-itertools package if you want. This also lets you get rid of the try/except block (which probably didn't affect runtime much, but it helps code clarity).
Python has a built-in function, max, to find the maximum value of an iterable. If you use it, then the nuts and bolts of the loop as well as the compare-and-store-if-greater process get handled internally by the interpreter, which should be faster than doing them manually in pure Python.
Repeatedly accessing an attribute of an object, namely the method self._suffix_str_array, is slower than accessing it once and storing it locally as a new variable.

Let's check the effect of these changes on the runtime of the driver program:

lcs     0.49046793299203273
optlcs1 0.4739605739887338
optlcs2 0.4895538759883493
optlcs3 0.4828952929965453

Not much of a change. Actually, using max is slower here, so let's discard that change.

Time now to take a closer look at commonprefix.

If you look at the source code of this function,

def commonprefix(m):
    "Given a list of pathnames, returns the longest common leading component"
    if not m: return ''
    s1 = min(m)
    s2 = max(m)
    for i, c in enumerate(s1):
        if c != s2[i]:
            return s1[:i]
    return s1

you see that it goes through some preliminary steps relating to the fact that it needs to handle a list of potentially several paths. You only ever have two strings to compare, so you can skip that - and in fact, if you look back at the profiling data, you'll see that these calls to max and min do take up a significant amount of time. Therefore, it makes sense to implement your own version of commonprefix without the max and min calls.

This makes a big difference in the runtime, cutting it down by over 30%:

lcs     0.49046793299203273
optlcs3 0.4828952929965453
optlcs4 0.3215277789859101

If you think about it, you don't even really need the common prefix itself, except for the one string you actually return from _get_lcsubstr. You only need its length, so you can decide which substring is the longest. So instead of using a function that finds the full common prefix, just write one that will give you its length. You store the length, along with one of the strings, and at the end of _get_lcsubstr, use the length to trim the stored string.

def _get_lcsubstr(self):
    # initialize
    for s1, s2 in mt.pairwise(suffix_str_array):
        if s1[-1] != s2[-1]:
            substr_len = get_common_prefix_length(s1, s2)
            if substr_len > max_len:
                max_len = substr_len
                max_substr = s1
    return max_substr[:max_len]

This shaves another few percent off the runtime:

lcs     0.49046793299203273
optlcs4 0.3215277789859101
optlcs5 0.30241094100347254

If it's faster to avoid computing a substring in your commonprefix replacement, you might think of doing the same thing when you're finding the suffixes in the first place. In other words, instead of calculating all the suffixes of a string in _get_suffix_str, just make a list of (index, which_string) tuples to represent the suffixes. Then whenever you need to actually compare two suffixes, instead of taking a substring of the original string, you just start comparing characters at the required indices.

There are two problems with this in practice: first, Python isn't well suited to iterating from an arbitrary point in the middle of a string. In a language like C, where strings are character pointers, this would work out quite well, because you can jump into the middle of the string by advancing a pointer. But in Python, iterating from the middle of a string requires you to either start from the beginning and just skip the first several characters, or bypass the whole iteration mechanism and use a for loop with an integer index to access characters inside the string by their indices (which typically involves more Python code that is relatively inefficient). And besides, the other reason is you need to create the substrings anyway to sort them. If you try to do it so that you use the substrings as comparison keys without actually storing them, the program spends a lot of time converting between a substring and its index.

All told, the changes you need to make to the code to use indices everywhere wind up hurting, not helping. Here's the timing result:

lcs     0.49046793299203273
optlcs5 0.30241094100347254
optlcs6 0.3683815289987251

At this point, you can spend a lot of time making little tweaks to try to squeeze some extra performance out of the program, but I don't think there are any major performance gains left. It's already something like 40% faster than the original, which is not bad.

Of course, as dawg wrote in a comment, you can accomplish this task using Python's standard module difflib.

def difflib_longest_match(a, b):
    i, j, k = difflib.SequenceMatcher(a=a, b=b
        ).find_longest_match(0, len(a), 0, len(b))
    return a[i:i+k]

I have no idea what's in the difflib code (well, I could look, but I'll leave that as an exercise), but it's clearly heavily optimized for this kind of task. It's another 25% faster than my best version of your program:

lcs     0.49046793299203273
optlcs5 0.30241094100347254
difflib 0.2154458940058248

So if you really want to do this not for educational purposes, just use difflib.

Here is the content of the test script I used.

import difflib
import itertools as it
import more_itertools as mt
from os.path import commonprefix

class LongestCommonSubstr(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]

    def _get_lcsubstr(self):
        try:
            substr_len =0
            max_len = 0
            lcs = None
            for i,n in enumerate(self._suffix_str_array):
                if n[-1] != self._suffix_str_array[i+1][-1]:
                    substr = commonprefix([n,self._suffix_str_array[i+1]])
                    substr_len = len(substr)
                    if substr_len > max_len:
                        max_len = substr_len
                        lcs = substr

        except IndexError:
            pass
        return lcs

class OptimizedLongestCommonSubstr1(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]

    def _get_lcsubstr(self):
        substr_len =0
        max_len = 0
        lcs = None
        for s1, s2 in mt.pairwise(self._suffix_str_array):
            if s1[-1] != s2[-1]:
                substr = commonprefix([s1,s2])
                substr_len = len(substr)
                if substr_len > max_len:
                    max_len = substr_len
                    lcs = substr
        return lcs

class OptimizedLongestCommonSubstr2(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]

    def _get_lcsubstr(self):
        return max((commonprefix([s1,s2]) for s1, s2 in mt.pairwise(self._suffix_str_array) if s1[-1] != s2[-1]), key=len)

class OptimizedLongestCommonSubstr3(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]

    def _get_lcsubstr(self):
        s_array = self._suffix_str_array
        return max((commonprefix([s1,s2]) for s1, s2 in mt.pairwise(s_array) if s1[-1] != s2[-1]), key=len)

class OptimizedLongestCommonSubstr4(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]
    @staticmethod
    def _get_common_prefix(s1, s2):
        for i, c in enumerate(s1):
            if c != s2[i]:
                return s1[:i]
        return s1

    def _get_lcsubstr(self):
        s_array = self._suffix_str_array
        gcp = self._get_common_prefix
        substr_len =0
        max_len = 0
        lcs = None
        for s1, s2 in mt.pairwise(s_array):
            if s1[-1] != s2[-1]:
                substr = gcp(s1, s2)
                substr_len = len(substr)
                if substr_len > max_len:
                    max_len = substr_len
                    lcs = substr
        return lcs

class OptimizedLongestCommonSubstr5(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring+'0'
        self.rstring = rstring+'1'
        self._suffix_str_array = sorted(self._get_suffix_str(self.lstring)
                                    + self._get_suffix_str(self.rstring))
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_suffix_str(string):
        return [string[i:] for i in range(len(string))]
    @staticmethod
    def _get_common_prefix_length(s1, s2):
        for i, c in enumerate(s1):
            if c != s2[i]:
                return i
        return len(s1)

    def _get_lcsubstr(self):
        s_array = self._suffix_str_array
        gcpl = self._get_common_prefix_length
        max_len = 0
        max_substr = ''
        for s1, s2 in mt.pairwise(s_array):
            if s1[-1] != s2[-1]:
                substr_len = gcpl(s1, s2)
                if substr_len > max_len:
                    max_len = substr_len
                    max_substr = s1
        return max_substr[:max_len]

class OptimizedLongestCommonSubstr6(object):
    def __init__(self, lstring, rstring):
        self.lstring = lstring
        self.rstring = rstring
        self._suffix_str_array = sorted(
            [(i, True) for i, _ in enumerate(lstring)]
            + [(i, False) for i, _ in enumerate(rstring)],
            key=lambda t: (lstring if t[1] else rstring)[t[0]:])
        self.longest_common_substr = self._get_lcsubstr()

    @staticmethod
    def _get_common_prefix_length(s1, start1, s2, start2):
        for i, c in enumerate(it.islice(s1, start1, None)):
            if c != s2[i + start2]:
                return i
        return i

    def _get_lcsubstr(self):
        s_array = self._suffix_str_array
        gcpl = self._get_common_prefix_length
        max_len = 0
        max_start = 0
        max_substr = ''
        for (i1, b1), (i2, b2) in mt.pairwise(s_array):
            if b1 != b2:
                if b1:
                    s1 = self.lstring
                    s2 = self.rstring
                else:
                    s1 = self.rstring
                    s2 = self.lstring
                substr_len = gcpl(s1, i1, s2, i2)
                if substr_len > max_len:
                    max_len = substr_len
                    max_start = i1
                    max_substr = s1
        return max_substr[max_start:max_start+max_len]

def lcs_longest_match(a, b):
    return LongestCommonSubstr(a, b).longest_common_substr

for n in it.count(1):
    if 'OptimizedLongestCommonSubstr{}'.format(n) in globals():
        exec('''def optimized_lcs_longest_match_{}(a, b):
    return OptimizedLongestCommonSubstr{}(a, b).longest_common_substr
'''.format(n, n))
    else:
        break

def optimized_lcs_longest_match(a, b):
    return OptimizedLongestCommonSubstr(a, b).longest_common_substr

def difflib_longest_match(a, b):
    i, j, k = difflib.SequenceMatcher(a=a, b=b).find_longest_match(0, len(a), 0, len(b))
    return a[i:i+k]

if __name__ == '__main__':
    import timeit, sys
    n_runs = 10000
    print('lcs    ', timeit.timeit('lcs_longest_match(sys.argv[1], sys.argv[2])', setup='from __main__ import lcs_longest_match', number=n_runs))
    for k in range(1, n):
        print('optlcs{}'.format(k), timeit.timeit(
            'optimized_lcs_longest_match_{}(sys.argv[1], sys.argv[2])'.format(k),
            setup='from __main__ import optimized_lcs_longest_match_{}'.format(k),
            number=n_runs))
    print('difflib', timeit.timeit('difflib_longest_match(sys.argv[1], sys.argv[2])', setup='from __main__ import difflib_longest_match', number=n_runs))

    print('control results:', difflib_longest_match(sys.argv[1], sys.argv[2]))
    for k in range(1, n):
        print('test results {}: '.format(k), eval('optimized_lcs_longest_match_{}(sys.argv[1], sys.argv[2])'.format(k)))


1. Arraymake:

Change the name to MakeArray as it makes more sense or name it MakeArrayFromInput to make it more descriptive to what it actually does
Name the parameter length, as in : int* MakeArray(size_t length) - this
is descriptive and you don't have to put the comment explaining what it does
Name local variables in camelCase, so array instead of Array
Matter of preference but consider declaring array as int *array instead of int* array - you won't forget that the variable itself is a pointer and you have to declare additional pointers in the same statement with *s - int *array, *anotherPointer;
int number ,i; //numbers filling the array - comment is redundant and you don't have to declare i here, you can do it in the for - speaking of which:
for(i=0;i<(ln);i++) can become for(int i = 0; i < length; i++) - parentheses over ln are redundant, you should use better spacing to make it more readable.

2. Median:

The same story with the parameter and naming, also pass the array first, then the length : void Median(int *array, size_t length)
You don't have to declare the median variable before the if
Parentheses in the if are redundant : if(length % 2 == 0)
The calculation of the median in the case of an even number of elements is flawed - you do an integer division as both operands of / are ints - you'd have to make at least one of them a float e.g. like this median= (Array[(ln)/2]+Array[(ln/2 )-1] )/2.0; - see the 2.0 - this is a double literal, a floating-point number, not an int
You can shorten the calculation a bit by changing array[(length - 1) / 2]; to array[length / 2]; in the case of an odd number - the integer division will truncate the result into what you need
The function should do one thing - you're calculating the median and printing it in the same function. Consider making the function Median return the value of the median and print it somewhere else

The function then should look more or less like this :

float Median(int *array, size_t length)
{
    if(length % 2 == 0)
    {
        float median = (array[length / 2] + array[length / 2 - 1]) / 2.0;
        return median;
    }
    else
    {
        float median = array[length / 2];
        return median;

        //or just simply
        return array[length / 2];
    }
}

Something more complex, the function can be greatly simplified by using the ternary operator ?: :

float Median(int *array, size_t length)
{
    return length % 2 == 0 ?
           (array[length / 2] + array[length / 2 - 1]) / 2.0 :
           array[length / 2];
}

However, this is just trivia and you should aim to make your code as readable as possible

3. main

I think that all of the comments are redundant here, everything is self-documenting
Naming : ArResult should be something descriptive and simple - the most simple way would be to name it just array
Spacing in function parameters e.g.
scanf("%Iu", &count); instead of scanf("%Iu",&count);

4. General advice

Work on the spacing - try to put spaces between operands and operators as in if(length % 2 == 0), in places like for loops : for(int i = 0; i < length; i++) and in function parameters : scanf("%Iu", &count);
Put comments before the thing they're describing, not at the end of the line - this way if you want to make it a multi-line comment you won't have problems and it's more readable that way e.g.

like this:

//Function for creating array of numbers.
int* MakeArray(size_t);
//Function for calculating the median.
int Median(int*, size_t);

This is just an example - I think the function names are descriptive enough not to put comments there, they will clutter the view :

int* MakeArray(size_t);
int Median(int*, size_t);

EDIT

What I didn't initially remember is that to calculate the median, you have to use a sorted array. You should probably do that separately

int *array = MakeArray(count);
qsort(array, count, sizeof(int), compare)
float result = Median(count, array);

The qsort is a function from stdlib.h. You have to also declare a function compare to use in it e.g.

int compare(const void* a, const void* b)
{
     int int_a = *((int*) a);
     int int_b = *((int*) b);

     if(int_a == int_b) return 0;
     else if (int_a < int_b) return -1;
     else return 1;
}

This is taken from https://stackoverflow.com/a/3893967/7931009


Rounding a number up to the nearest multiple of a power of 2

Function interface

First of all, it is good that you used a function to contain the code solving the problem!

However, there are some shortcomings:

It is very confusing that the function parameter representing \$n\$ from the problem description is named x, and the one representing \$x\$ is named y.
The function is badly named, as it doesn't multiply the two arguments. A better name would be smallest_multiple or similar.
The function lacks a docstring which describes what it does. It is therefore hard to check if it is implemented correctly. The first sentence of the problem description would make a good docstring (replacing "print out" with "return").

Bug

The function returns the wrong result if both arguments are the same, for example:

>>> multiply(8, 8)
16

But the smallest multiple of 8 which is greater than or equal to 8 should be 8, not 16.

(Maybe you thought that 8 is not considered a multiple of 8?)

The bug is fixed by removing the res != x condition.

Loop exit condition

Let's ignore everything that doesn't affect the max_multiple_try variable:

max_multiple_try = 0
while max_multiple_try != float("inf"):
    …
    if …:
        …
    else:
        max_multiple_try += 1

We can see that it is always an integer (which has unlimited size in Python). Therefore it can never be equal to the special float('inf') value, and the loop could as well have been written using while True: ….

However, as max_multiple_try simply counts the loop iterations, this pattern would be better written as a for loop using itertools.count (and using a simpler and more descriptive name):

from itertools import count
…
for factor in count():  # factor = 0, 1, 2, …
    res = x * factor
    if res >= y:
        return res

Top-level code

Congratulations on using if __name__ == '__main__' and using with to open a file!

Here are some improvements:

Iterate directly over data instead of using readlines.
Use tuple assignment:

x, n = line.rstrip().split(",")
print(multiply(int(n), int(x)))

Alternative algorithm

The problem states that the value of \$n\$ is constrained to powers of two. While your solution actually solves the more general problem for any value of \$n\$, this description probably aims for a different algorithm which makes use of the constraint.

Hint: What does the binary representation of powers of two look like? How could you solve the problem if \$n\$ were constrained to powers of ten?


Disclaimer: Not a Code Reviewer

Your solution looks fine;
Overall, we would want to stay away from regular expressions in solving algorithm problems – especially easy LeetCode questions are to measure our implementation skills;
C# has two simple functions that would do the job of (\W|_):

public class Solution {
    public bool IsPalindrome(string s) {
        var left = 0;
        var right = s.Length - 1;

        while (left < right) {
            if (!char.IsLetterOrDigit(s[left])) {
                ++left;

            } else if (!char.IsLetterOrDigit(s[right])) {
                --right;

            } else {
                if (char.ToLower(s[left]) != char.ToLower(s[right])) {
                    return false;
                }

                ++left;
                --right;
            }
        }

        return true;
    }
}

We can just a bit improve the runtime, but not so much, using a more readable expression such as:

(?i)[^a-z0-9]+;

(?i) matches the remainder of the pattern based on the insensitive flag;

[^A-Za-z0-9]+;

public class Solution {

    public bool IsPalindrome(string s) {

        s = System.Text.RegularExpressions.Regex.Replace(s, @"(?i)[^a-z0-9]+", "");

        if (s == null || s == "" || s.Length == 1 || s == " ") {
            return true;
        }

        for (int i = 0, j = s.Length - 1; i <= s.Length / 2  && j >= s.Length / 2; i++, j--) {
            if (Char.ToLower(s[i]) == Char.ToLower(s[j])) {
                continue;
            }

            return false;
        }

        return true;
    }
}

RegEx Circuit
jex.im visualizes regular expressions:

Demo
If you wish to simplify/update/explore the expression, it's been explained on the top right panel of regex101.com. You can watch the matching steps or modify them in this debugger link, if you'd be interested. The debugger demonstrates that how a RegEx engine might step by step consume some sample input strings and would perform the matching process.

Happy Coding! ( •ˆ_ˆ• )


Optimizing Project Euler 36 - double-base palindromes

In any programming language, I'd consider (assuming c is a condition with a boolean value):

if c:
    return True
else:
    return False

or it's equivalent :

if c:
    return True
return False

to be an anti-pattern that should be written:

return c

Your for loop can easily be rewritten into a more concise (and more efficient according to my benchmarks) way using sum :

return sum(n for n in range(limit) if is_palindrome(n) and is_palindrome(int(bin(n)[2:])))

An easy optimisation is not to compute str(n) more than needed :

def is_palindrome(n):
    s = str(n)
    return s == s[::-1]

Also, it might even be faster to feed strings to your functions :

def is_palindrome(s):
    return s == s[::-1]
return sum(n for n in range(limit) if is_palindrome(str(n)) and is_palindrome(bin(n)[2:]))

On my machine, running your function 10 times, the running time went from 470 ms to 310 ms per iterations.


Well if you want to use the Object prototype here is how I would structure it:

// uses Object.extend for the constructor https://gist.github.com/rlemon/5256375

function Query(options) {
  this.value; // I would store the value in the object itself. Otherwise you should use this as a utility object and not deal with prototype.
  // other defaults
  Object.extend(this, options); // extend options to the object
}

Query.prototype = {
  constructor: Query,
  removeStopWords: function() { .. return this;},
  stemmer: function() { .. return this;},
  convertBoolean: function() { .. return this;} // returns itself for chaining
};

var q = new Query({ value: ' Foo Bar Hello World ' });
q.removeStopWords().stemmer().convertBoolean();
alert(q.value);

But this is how I would structure it based on how I am assuming you are using this. If the use cases changed or the requirements were not fully understood I would possibly write it a number of other ways.


Using Java to implement a hash table (dictionary) with only arrays

Do not special case an immediate success. The test for table[code] == null in

    if (table[code] == null) {
        table[code] = new String[]{key, value};
    } else {
        while (table[code] != null) {
        ....

does not do anything useful, but only adds noise. Dive into the loop immediately.

Ditto for table[code][0].equals(key) in remove and get.
Such low level utility functions shall not System.out.println (in any case, an error message shall go to System.err). Prefer returning an error code, and let the caller decide what to do.
It is unclear how well String.hashCode() is suited for this design of a hash table. Depending on the table size, you may get plenty of collisions. Using Math.abs on a hash code also biases the distribution. Need to measure.


Your initial version - 6.834 seconds.

Version with quot and even operations replaced with bitwise - 6.651 seconds.

This version - 0.928 seconds:

import Data.Bits

tOdd sum 1 = sum
tOdd sum x = tEven (sum + (tEven2 0 x)) (x - 1)

tEven sum x = tOdd (sum + (tOdd2 0 x)) (x - 1)

tEven2 sum x | (x .&. 2) == 0 = tEven2 (sum + 1) (x `shiftR` 1)
tEven2 sum x | otherwise      = tOdd2 (sum + 1) (x `shiftR` 1)

tOdd2 sum 1 = sum
tOdd2 sum x = tEven2 (sum + 1) (3 * x + 1)

main = print . show $ tEven (0 :: Int) (3000000 :: Int)

The previous with -O3 -fllvm -optlo-O3 flags - 0.686 seconds.

The previous with all functions supported with INLINE pragma - 0.604 seconds.

More than 10 times faster than initially :)

Yes, low-level numeric operations and iterations are hard in Haskell, the language doesn't suite well for this.


Is there a way to make reshaping arrays shorter and cleaner?

You took a curved path to where you want to go :)

It seems you are ill at ease at building flat lists and take long routes at building nested ones then unpacking them. But you can replace :

    omegas_ext = []
    for element in omegas:
        item = [x for x in element if x is not None]
        omegas_ext.append(item)
# [...]
    for element in omegas_ext:
        for item in element:
            omegas_unpacked.append(item)

By:

    omegas_ext = []
    for element in omegas:
        item = [x for x in element if x is not None]
        omegas_ext.extend(item)

Second, this if else is unecessary. [value] * 1 is equivalent to [value].

        if len(omegas_ext[idx])>1:
            value = delays[idx]
            item_to_add = [value]*(len(omegas_ext[idx]))
            delays_exp.append(item_to_add)
        elif len(omegas_ext[idx]) == 1:
            value = delays[idx]
            delays_exp.append([value])

Can be replaced by:

    if len(omegas_ext[idx])>=1:
        delays_exp.append([delays[idx]]*(len(omegas_ext[idx])))

Again you unpack this later on. So you could extend and this would also make it unecessary to have this check for an element (since extending with an empty list is a no-op)

    delays_exp.extend([delays[idx]*len(omegas_ext[idx]))
    # itertools.chain no longer needed

Finally, you missed the fact you can build the two lists in a single for, by using zip. This would save you the need to use an index to recompute the length of the omegas_ext items.

Here would be the function :

def spp_method(omegas, delays):
    delays = delays[delays != np.array(None)]
    omegas_ext = []
    delays_exp = []
    for delay, element in zip(delays, omegas):
        item = [x for x in element if x is not None]
        omegas_ext.extend(item)
        delays_exp.extend(len(item) * [delay])
    return np.array(delays_exp), np.array(omegas_ext)

Code returns the same output.


General style points

No need to split the str into an array, you can access the character directly using bracket notation. eg str[i] === ">"

If you want a number to cycle over a range use the remainder operator % to do that. Example  to add 1 memory[pointer] = (memory[pointer] + 1) % 255 and to subtract memory[pointer] = (memory[pointer] + 254) % 255.
However as you want to simulate memory as bytes you may as well use a typed array that has an Unsigned Char. The bytes in the array behave the same as they would on the VM that runs BF commands. Example const memory = new Uint8Array(1024)

Not sure why you throw overflow for "<" when you don't throw the same error >. < should just cycle to highest byte in memory.

Use "const" for variables that do not change.

Use === and !== rather than == and !=

Don't double space (every other line empty). It makes the code hard to read as it will not fit to one screen.

Naming is rather poor.
-result could be better as "output"

str maybe commands or program
i maybe programCounter or commandIndex. I am old school so would use pc

Rather than index into an array over and over, as you do with array[i], store it in a variable const command = array[i]; and then if(command === ">") { makes the code more compact and easier to follow.

Design.
You can use a lookup to get the action required for each command. (see example commandList) rather than one long if() {}else if().... This is a lot quicker to run as you don't need to step over else if statements to find the matching block. Also make it easy to add or change commands, and easier to read.
The Bug
As I pointed out in the comments there is a bug. You have not implemented the "[" command correctly. This is a big issue as many programs will fail. To solutions are non trivial.
There are two solutions.

When you need to skip ahead search for the matching close. This can cost a lot of performance.
Compile the code and locate matching open and close [ and ] so that you can quickly locate the match and move to it if required.

Ease of use
The input and output would be better as strings rather than arrays. Makes the BF easier to use.
The halting problem.
There is a fundamental requirement you have missed. You can never know if the program you are running will exit.
As JavaScript is blocking this means there is no way to stop the execution if the bf code goes into infinite loop. You should protect against this problem.
The easiest is to put a limit on the number of instructions, and throw error if reached.
Or you could have the commands execute on a timer.
Example
The function name is bfVM for brainFuckVirtualMachine and requires a compiled program. (as that is how I fixed your bug).
I included the compiler, all it does is find matching blocks and maps them to each other. returns a compiled object.
And two programs as testing fodder.

// Run two comnpiled BF programes.
setTimeout(() => {
console.log(bfVM(helloWorld))
console.log(bfVM(addNums))
},0);
/* Old school naming pc is program counter, ptr is a pointer, and
   inPtr is input pointer */
function bfVM(compiled, input) {
    const program = compiled.program;
    const blocks = compiled.blocks;
    const RAM = compiled.RAM;
    const memory = new Uint8Array(RAM);
    const commandList = {
        ">"(){ ptr = (ptr + 1) % RAM },
        "<"(){ ptr = (ptr + RAM - 1) % RAM },
        "+"(){ memory[ptr]++ },
        "-"(){ memory[ptr]-- },
        "."(){ output += String.fromCharCode(memory[ptr]) },
        ","(){ memory[ptr] = input.charCodeAt(inPtr++) },
        "["(){ pc = !memory[ptr] ? blocks.get(pc)[0] : pc },
        "]"(){ pc = memory[ptr] ? blocks.get(pc)[0] : pc  },
    };

    var cycles = 0, ptr = 0, pc = 0, inPtr = 0, output = "";
    while (pc < program.length && cycles++ < MAX_CYCLES) {
        const cmd = commandList[program[pc]];
        cmd && cmd();
        pc ++;
    }
    if (cycles=== MAX_CYCLES) { throw new Error("Cycle limit reached at PC: " + pc)  }
    return output;
}

const MAX_CYCLES = 1024 * 1024;  // Limit of execution steps.
function compileBF(program, RAM) {
    var pc = 0;
    const blocks = new Map();
    const blockStack = [];
    while (pc < program.length) {
        if (program[pc] === "[") {
            blocks.set(pc, [])
            blockStack.push(pc);
        } else if(program[pc] === "]") {
            const open = blockStack.pop();
            const block = blocks.get(open);
            if (block === undefined) {
                throw new Error("Syntax error: No matching block for ']' at " + pc);
            }
            block[0] = pc;
            blocks.set(pc, [open]);
        }
        pc++;
    }
    if (blockStack.length) {
       throw new Error("Syntax error: Block missmatch at " + pc);
    }
    return {program, blocks, RAM};
}

const nums = ["", "+", "++", "+++", "++++", "+++++", "++++++", "+++++++", "++++++++", "+++++++++"]
const toASCIINum = "++++++++[<++++++>-]<"
const helloWorld = compileBF(("++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++."), 14)
const addNums = compileBF(nums[5]+">"+nums[4]+"[<+>-]"+toASCIINum+".",2);


Correctly applying the "rule of five" to a RAII socket wrapper

These are good questions!

1. You should indeed have the copy constructor & assignment operator deleted.

The "rule of five" tells you to specifically define a copy c'tor and assignment operator - but it doesn't tell you that you have to make the available. It is a perfectly valid choice to decide to not allow your object to be copied or non-move-assigned - only moved. An example of this: std::unique_ptr.

It's also what I would recommend in your case, because:

Like you said, duplicating a file descriptor is weird and unexpected.
It's not obvious to the user of your class what the copy behavior should be.
There don't seem to be - AFAICT - common scenarios in which you would copy, rather than move or pass by reference, a TCP socket.

2. Consider using std::optional to indicate "no valid value" or "missing" or "none"

In C (and the C system call bindings on Unix-like systems), it is a convention to use the -1 value for an invalid/missing file descriptor is a convention. We know that the int type is actually larger than the actual space of possible file descriptor values, so we use a junk value, which we assume the OS never uses, to indicate "no valid value". Now, this works fine; and you could choose to, say, define a static class constant:

static constexpr const int no_file_descriptor { -1 };

and then write:

close(_s);
_s = other._s;
other._s = no_file_descriptor;

As @HolyBlackCat suggests, you must get _s to be closed somehow. You could also just swap the two descriptors, but I find that to contradict the element of least surprise.

but you might want to consider the more general (though less space-efficient) solution, which is the std::optional<T> type template. It is intended for exactly your case: Either holding some value of type T (in your case, int), or holding some indication of "no value". Using an optional, you could write:

close(_s.get());
_s = other._s;
other._s = std::nullopt;

You'll still need to write your move assignment and move construction code, unfortunately (thanks @CassioRenan for noticing these are both necessary).

Other suggestions

Don't use a plain int; either find a type definition of a file descriptor from some library you're using, or if you have no definition to borrow, have using file_descriptor = int; or using file_descriptor_index = int.
_s is a bad member name. Use something more explicit, e.g. descriptor_index_ or posix_descriptor_index_.
Always check the return value of library/system calls! And handle errors.

The modified code

class tcp_socket {
protected:
    void close_if_neccessary() {
       constexpr auto socket_close_failed { -1 };
        if (descriptor_index_.has_value()) {
            auto retval = close(descriptor_index_.value());
            if (retval == socket_close_failed) {
               // throw something here, e.g.:
               // throw std::system_error(errno, std::system_category(), "close()");
            }
    };
public:
    using file_descriptor_index = int;

    tcp_socket() : descriptor_index_() {
       constexpr auto socket_creation_failed { -1 };
       auto retval = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
       if (retval == socket_creation_failed) {
           // throw std::system_error(errno, std::system_category(), "socket()");
       }
       descriptor_index = retval;
    };

    tcp_socket(const tcp_socket& other) = delete;
    tcp_socket& operator=(const tcp_socket& other) = delete;
    tcp_socket& operator=(tcp_socket&& other) noexcept {
        if (other.descriptor_index_ != descriptor_index_) {
            close_if_neccessary();
            descriptor_index_ = other.descriptor_index_;
            other.descriptor_index_ = std::nullopt;
        }
        return this;
    };
    tcp_socket(tcp_socket&& other) noexcept :
        descriptor_index_(other.descriptor_index_)
    {
        other.descriptor_index = std::nullopt;
    }
    ~tcp_socket() {
        // You might want to wrap this in a try-catch, since
        // destructors shouldn't throw. Otherwise you're risking
        // a double-exception and immediate program termination.
        close_if_necessary();
    }

protected:
    std::optional<file_descriptor_index> descriptor_index_;
};

See this StackOverflow question about the weird exception code in the comments:


cs50.h?

This seems less like a set of "training wheels" and more like a bicycle for fish. It's potentially confusing, opaque, and doesn't seem all that useful. If I were you, I'd be learning how to code in real C - using char*, and bool from stdbool.h.

Don't store things that should be computed

Your string alphabet shouldn't exist. Just iterate a char between a-z and A-Z. Characters can be incremented the same way that integers can.

Input validation

It seems like you expect hash to be two characters long, but you don't check that. You should be checking it with strlen; then you can issue memcpy without later setting a null terminator, as it'll be null-terminated already.

DRY

Don't repeat yourself. This is the most important aspect of the program that needs improvement. This block:

                for(int m = 0; m < alphabet_len; m++)
                {
                    password[4] = alphabet[m];
                    if(!strcmp(hash, crypt(password, salt)))
                    {
                        flag = true;
                        break;
                    }
                }
                if(flag)
                    break;

is repeated nearly verbatim five times. There are many different ways to condense this. The easiest is probably a recursive function that calls itself with an increasing depth integer. This may actually decrease the performance of the application, but that's up to you to test. There are also ways to rewrite this loop to have state so that neither copy-and-paste nor recursion are necessary; you'll probably want to compare such a method against a recursive method to see which is more performant and clean.


Produce the sum of all white space separated integers in a given file

You collect all numbers but then use the collection only for the sum. Better just update the sum. And I find the whole trying int and if fails use word a bit complicated. How about just always reading words and then reading numbers from them? (The >> writes zero if there's no number.)
#include <fstream>
#include <iostream>
#include <sstream>
#include <string>
using namespace std;

int main() {
    int sum = 0;
    ifstream file("file1.txt");
    string word;
    while (file >> word) {
        int number;
        istringstream(word) >> number;
        sum += number;
    }
    cout << sum << endl;
}


I want to call SaveChangesAsync and when this succeeds, clear a list.

You don't need ContinueWith, you can just write it like this

public async Task<int> CommitAsync()
{
    var result = await DataContext.SaveChangesAsync();
    _addedEntities.Clear();
    _changedEntities.Clear();
    _deletedEntities.Clear();
    return result;
}

In the original code, the catch block will never run.

Consider what happens when SaveChangesAsync throws a DbEntityValidationException: antecedent.Result will throw an AggregateException wrapping the DbEntityValidationException.

It's poor practice to throw an Exception as it means client code cannot be specific about the exceptions it catches. I would suggest changing RethrowException to a method that constructs the exception message you want, then calling it like this:

public async Task<int> CommitAsync()
{
    try
    {
        var result = await DataContext.SaveChangesAsync();
        _addedEntities.Clear();
        _changedEntities.Clear();
        _deletedEntities.Clear();
        return result;
    }
    catch (DbEntityValidationException e)
    {
        var exceptionMessage = GetDbEntityValidationExceptionMessage(e);
        throw new DbEntityValidationException(exceptionMessage, e);
    }
}

This also removes the return default(int); which is just there to appease the compiler.


SpamAssassin spam analyzer script in PHP based on sa-learn command

Here's an untested Bash version:

#!/usr/bin/env bash
set -o errexit -o noclobber -o nounset

while IFS= read -r -u 9 path
do
    /usr/bin/sa-learn --spam "$path" \
        > "/home/domainexample.ru/.spamassassin/logs/$(date +%d.%m.%Y-%G:%S)_analyzer.log" \
        2>&1 &
    sed -i -e 's/^Subject:/Subject: [SPAM] /' "$path"
    rm "$path"
done 9< <(grep -FL '[SPAM]' /home/domainexample.ru/Maildir/.Junk/cur/* 2> /dev/null)

Some changes from the original:

Uses grep's -F option to speed up search.
Runs sa-learn repeatedly instead of once to avoid having to accumulate the data. Shouldn't slow down the execution since the processes are backgrounded.
Deletes files as soon as they are processed, to avoid reprocessing if the previous run failed.


VBA Text Array - Scan two columns rather than one

Consistent Indentation.
The first thing I would address, is the indentation. Improving code that isn't properly formatted is just... not right.
Here are the signs:
Sub arraycolumnmatch()
Dim txtArray As Variant, T As Variant
Dim I As Long, J As Long

And:
Next I
End Sub

But also:
 'Check for surnames which could match first names such as Mary Michael
       If Not Range("G" & I).Value = T Then

The key is consistency - here's your code, with consistent indentation (comments omitted). Notice each code block (If...End If, For...Next, etc.) consistently adds an indentation level, and the end of the block lines up with its beginning - also notice that indentation levels are consistently 4 spaces wide; that's the default VBE setting for the Tab key:
Sub arraycolumnmatch()
|
|   Dim txtArray As Variant, T As Variant
|   Dim I As Long, J As Long
|
|   For I = 2 To Range("E50000").End(xlUp).row
|   |   typ = Range("F" & I).Value
|   |   If typ = "" Then
|   |   |   txt = Range("J" & I).Value
|   |   |   txtArray = Split(txt, " ")
|   |   |
|   |   |   For Each T In txtArray
|   |   |   |   For J = 2 To Range("G50000").End(xlUp).row
|   |   |   |   |   If Range("G" & J).Value = T Then
|   |   |   |   |   |   match_txt = T
|   |   |   |   |   |   Range("G" & I).Value = match_txt
|   |   |   |   |   |   Exit For
|   |   |   |   |   End If
|   |   |   |   Next J
|   |   |   Next T
|   |   |
|   |   |   For Each T In txtArray
|   |   |   |   For J = 2 To Range("F50000").End(xlUp).row
|   |   |   |   |   If Range("F" & J).Value = T Then
|   |   |   |   |   |   match_txt = T
|   |   |   |   |   |   If Not Range("G" & I).Value = T Then
|   |   |   |   |   |   |   Range("F" & I).Value = match_txt
|   |   |   |   |   |   |   Exit For
|   |   |   |   |   |   End If
|   |   |   |   |   End If
|   |   |   |   Next J
|   |   |   Next T
|   |   End If
|   Next I
|
End Sub

Much easier to follow, isn't it?

Meaningful Names.
Next thing I would address is the naming. Improving code where you have to quadruple-check every change you make because you're not sure you're changing the right thing is just not efficient. Using meaningful names for all identifiers fixes that.
Starting with the method's name itself:
Sub arraycolumnmatch()

Typically, VBA (VB in general) method names are PascalCase, for better readability. That would make it ArrayColumnMatch - but good method/procedure names should also start with a verb. What's this method doing exactly? should be answerable by simply looking at it's name.
The problem is, the method in question is doing too many things, so giving it a meaningful name is hard. But I'll get back to that.
Avoid chopping off identifers, like typ when you meant type - but then Type is a reserved keyword.. the solution isn't to make it less readable! I'm not sure which worksheet Range is referring to because Range is an implicit reference to Application.ActiveSheet (and that's very bug-prone!), so I'll just assume you meant to call it documentType.
txt is another meaningless name (heck, people use that as a prefix for TextBox controls!): it stands for that manual entry field you're trying to parse, right? How about targetFieldValue, or manualTextField?
Also avoid single-letter identifiers, like T - assuming txt is manualTextField, I'd go with manualTextFieldValue.
Local variables are typically named in camelCase, so I and J would be i and j - those are typically used for iterating loops as you're doing, so I'm not going to complain about those, except both loops seem to be iterating the very same rows on the active sheet (whatever that is), and they're nested, too... and since they're row numbers I'd typically name them xlRow[WhatItsFor].

Undeclared Identifiers
Always, systematically, consistently, automatically, thoughtlessly stick Option Explicit at the top of every VBA module. Don't question it, just do it: without it, you can have a bug by simply having a little typo in an identifier name. With it, your VBA code will refuse to compile and run if an identifier isn't declared anywhere. Use it. Always.
If match_txt is declared outside the scope of the procedure, then move the declaration inside that scope - it doesn't need to be at module scope. Always declare variables at the smallest possible scope; globals are evil.

vbNullString
A very minor point, but I have to mention it.
If typ = "" Then

Avoid using an empty string "" to mean no value. VBA defines the constant vbNullString specifically for that. And why should you bother typing vbNullString when "" does the same thing?
That's the thing. It doesn't do the same thing. "" is a full-fledged String value that needs its own allocated memory space. vbNullString is a null string pointer that's not allocated anywhere. Of course it's a very minor performance hit to allocate that string, but semantically, vbNullString is the right thing to do.
Don't believe me? Try this in the immediate pane (Ctrl+G):
?"Address for empty string: " & StrPtr("") & vbNewLine & "Address for vbNullString: " & StrPtr(vbNullString)

I get this output:
Address for empty string: 241665720
Address for vbNullString: 0

There's some kind of optimization going on under the hood though, so the pointer for "" will be reused every time it's encountered in your application's lifetime - as I said it's a very minor point. But just as you wouldn't declare an unused variable, you shouldn't allocate a memory address for a built-in constant value.

Twin Loops
You have this:
For Each T In txtArray

And then a few lines further, you have that:
For Each T In txtArray

You're iterating all elements of txtArray twice, when once would suffice. Also, wouldn't the last row of column "G" be the same last row as in column "F"? Wouldn't that last row be the same for every iteration of T? I'm asking, because your data seems to be laid out in a table with records, so you'd only need to find the row for the last record, whichever column you're using for that is irrelevant, no?
For J = 2 To Range("G50000").End(xlUp).row
...
For J = 2 To Range("F50000").End(xlUp).row

I'd extract a variable here, and assign it to Range("G50000").End(xlUp).row before the For Each T In txtArray loop starts, and then do
For J = 2 To xlLastRow

Then you're writing to G[i] if the value of G[j] matches the current txtArray element, and to F[i] if G[i] wasn't written to and the value of F[j] matches the current txtArray element.
Wouldn't that do the same thing then?
If Range("G" & j).Value = T Then
    Range("G" & i).Value = T
Else If Range("F" & j).Value = T Then
    Range("F" & i).Value = T
End If

Lastly, the J loop looks like it could be implemented as a Find instead of a loop.

Extract Methods.

The problem is, the method in question is doing too many things, so giving it a meaningful name is hard. But I'll get back to that.

So here we are. What's that method doing actually?
It...

finds the row number for the last record on the active sheet
iterates all records on the active sheet
splits a value in the current record into an array of values
iterates all values in that array
looks for a match of that value somewhere in columns G and F
writes that value to column G or F, whichever comes first

Extracting methods is a refactoring operation, and doing that manually in VBA is, honestly, a pain in the neck.
I've authored a refactoring tool that does exactly that (and more):

Of course I have a little bit of a vested interest (very little - it's free and open-source actually), but I'd recommend you take a look at what the Rubberduck VBE add-in can do for you.


Plotting the integer values in text format in matplotlib piechart

You got yourself in trouble by using plt.pie, and especially the keyword argument autopct, beyond its intended use.

The basic idea of the pie chart is to have the wedge labels outside the pie and perhaps percentages inside.

You wanted the wedge label and percentage inside the pie and manipulated the autopct keyword with a function to achieve this. This involved cumbersome calculations from percentages to values you already know.

Another solution could be to use the more simple labels keyword argument and change the resulting texts properties to be inside the pie instead outside, see code changes below:

contents = ['23 users Cluster_0', '21 users Cluster_1']

data = [int(x.split()[0]) for x in contents]

def pie_chart_labels(data):
    total = int(np.sum(data))
    percentages = [100.0 * x / total for x in data]
    fmt_str = "{:.0f}%\n({:d} users)"
    return [fmt_str.format(p,i) for p,i in zip(percentages, data)]

wedges, texts,  = ax.pie(data, labels=pie_chart_labels(data))

# shrink label positions to be inside the pie
for t in texts:
    x,y = t.get_position()
    t.set_x(0.5 * x)
    t.set_y(0.5 * y)

plt.setp(texts, size=10, weight="bold", color="w", ha='center')


It is nice that you have separate methods for throwing the dice and computing the counts. However, diceRoll can be simplified and counters supersimplified; like this

    public static int diceRoll() {
        Random rand = new Random();
        return rand.nextInt(6) + rand.nextInt(6) + 2;
    }

    public static int[] counters(int[] arr) {
        int[] counterArray = new int[11];

        for (int dice : arr) {
            counterArray[dice - 2]++;
        }

        return counterArray;
    }

(Edit Also, the name of each method should (preferably) begin with a verb: counters \$\rightarrow\$ countResults and diceRoll \$\rightarrow\$ rollDice.)

Hope that helps.


Longest words in dictionary that can be constructed from a list of letters

Preface

I am not sure the algorithm you have chosen is the best one. Sorting the letters, or using a frequency table of the letters, are obvious alternatives. However, this is code review, and not algorithm review.

So here we go:

Iterators

Don't use < on iterators, use != on iterators. < only works on bidirectional iterators, != works on all iterators. std::list<> is an example where < won't work.

Use ++it instead of it++. Theoretically, the latter has to return a temporary copy of the original, which is additional overhead when the iterator is a class. Practically, it won't make much of a difference (if any) as most compilers will inline and optimize. Still, it's one of the best signals whether someone is a true C++ expert or a converted C-programmer :P .

So

for (it = dict_words.begin(); it < dict_words.end(); it++)

Will become:

for (it = dict_words.begin(); it != dict_words.end(); ++it)

Predefined algorithms

Many of your one-line loops can be replaced with algorithms from <algorithm>.

In this small case, I wouldn't bother as it wouldn't help to make your code more clear or shorter.

On the other hand, experimenting with them in this case will make you more comfortable with them. With the introduction of lambdas in C++11, those algorithms will become much more common in new code!

C++11

Due to the fact that C++11 hasn't been around long yet (and compilers don't fully implement it yet), idiomatic C++11 will change over the next couple years. So take this section with a grain of salt.

That being said, use auto for the type of it. That part of C++11 has become idiomatic already.

E.g.:

vector<string>::iterator it;
for (it = dict_words.begin(); it != dict_words.end(); ++it)

will become:

for (auto it = dict_words.begin(); it != dict_words.end(); ++it)

EDIT: oops, forgot about the range-based for in C++11. See other answers for that. I'm not that well versed in C++11 yet, but I believe it would look like this:

for (auto it : dict_words)

More functions

Most of your code is in one long function, called main. Try to split it up.

Once you start inserting empty lines and comment headings to clarify what the block does, it's usually a good candidate to turn into a separate function.

E.g. this block of code (limited by empty lines, with a comment):

// Grab the dictionary.
fstream dict_file(argv[1]);
string in;
vector<string> dict_words;
while(getline(dict_file, in)) dict_words.push_back(in);
dict_file.close();

Can be replaced with this function:

void grab_dictionary(const char* filename, vector<string>& dict_words)
{
    fstream dict_file(filename);
    string in;
    while(getline(dict_file, in))
       dict_words.push_back(in);
}

Usage from main:

vector<string> dict_words;
grab_dictionary(argv[1], dict_words);

Additional benefit:

The comment is no longer needed - usually it has become the name of the function!
There is no need for dict_file.close() as the destructor will close the file at the right time.


First of all I don't think the time complexity is O(n!). You are doing DFS(unknowingly). So the time complexity would be O(n + m) and AFAIK you can't solve maze problem less than that unless you are using BFS. But nevertheless it will not make any difference.

Now your code has some bug, I think you didn't consider that way and it's ok if you are a beginner like me. Try to change the maze matrix to

int[][] m = {{1, 1, 1},
             {1, 0, 1},
             {1, 1, 0},
             {0, 1, 1}};

and you will get the picture. Maze can be solved but still it's not printing anything.
So what is the solution? BACK-TRACK.

Moreover you are only taking account of possibilities of going right and down, why not up and left?

You have hard-coded that the start point will be always (0,0) and end point will be (N-1, N-1)... yeah your puzzle says that but this will not be same for every puzzle right? So think of a general solving technique...

SPOILER ALERT

   Web resource

 My solution

Code Cleaning Tips

getMazePath method can be cleaned. In 1st if you are checking if you have reached the goal or not. Now move the checking condition to another method and return true if you have reached the goal.
In for loop(no need of this if you are using recursion) you are checking if you are in the maze and path exists or not... see how you are repeating yourself for every condition... think of a general solution.
No need for maintaining a Stack, since you are not using any of Stack's method but only add, which you can also use with a List.
Override toString method in Coordinate class. And get rid of long S.o.p s.

Good Luck!


How to increase download speed using the following method?

Forcing System.gc() isn't a good practice.
outputStream = new FileOutputStream(pdfFile);
        bufferedOutputStream = new BufferedOutputStream(outputStream);

should be wrapped into 1 variable to do a single close() afterwards:

outputStream = new BufferedOutputStream(new FileOutputStream(pdfFile));

All operations with outputStream should be put in try-finally block:

outputStream = new ...
try {
    outputStream.write(...);
    ...
} finally {
    outputStream.close();
}

Agree with the first comment: your file may simply not fit in memory. In SOAP API there should be some mean of getting the InputStream instead of an in-memory string.

Sorry for the formatting, couldn't fight the editor.

Good luck!


Printing text based on divisibility of number

In programming, there are only three useful numbers, 0, 1, and many.

If you write an if-condition that repeats all the tests already done, you're doing something wrong.

If you write similar ad hoc code for three special cases, you're doing something wrong.

E.g. adding "11 Plung" will require two more lines for the special case and the "no match" case will have to be made even longer and uglier.

Write simple code for a general case, and then supply specific data as needed:

def convert(number):
    names = [ [3, "Pling"], [5, "Plang"], [7, "Plong"] ]
    result = ""
    for value,message in names:
        if not number % value: result += message
    if message == "": message = f"{number}"
    return result

for index in range(1,25): print(index, convert(index))

That way, if you later need to add the "11" case, it's a simple change, and more importantly it happens in only one place.


Rather than use your code as a base, I chose to rewrite from scratch to incorporate a couple of ideas to get it to move diagonally properly, as well as hopefully make some cleaner code.

var speed = 2;

function animateTo(dom_elem, x, y, finishedCallback) {
    var pos = {
        x: dom_elem.offsetLeft,
        y: dom_elem.offsetTop
    };

    function animate() {
        var xToTarget = x - pos.x;
        var yToTarget = y - pos.y;

        xToTarget = Math.max(0, xToTarget);
        yToTarget = Math.max(0, yToTarget);

        if (xToTarget == 0 && yToTarget == 0) {
            finishedCallback();
        }

        var scale = speed/Math.sqrt(Math.pow(xToTarget, 2) + Math.pow(yToTarget, 2));

        var delta_x = xToTarget * scale;
        var delta_y = yToTarget * scale;

        pos.x += delta_x;
        pos.y += delta_y;

        pos.x = Math.min(x, pos.x);
        pos.y = Math.min(y, pos.y);

        dom_elem.style.left = pos.x + 'px';
        dom_elem.style.top = pos.y + 'px';

        setTimeout(animate, 1);
    }

    setTimeout(animate, 1);
}

var box = document.getElementById('box');
animateTo(box, window.innerWidth - box.offsetWidth, window.innerHeight - box.offsetHeight, function() {
    alert('done');
});

Firstly, I have abstracted it into the idea of animating a dom element to a particular position. You can then chain together the animations needed using the finishedCallback.

The trick to getting it to animate diagonally properly, instead of hitting the bottom and sliding along, is to move along x and y by different amounts depending on the ratio needed to reach the target instead of always incrementing by 1.

Imagine the current position of the box as one point on the hypotenuse of a triangle, and the destination as the other end of the hypotenuse. The distance along the x-axis forms one of the other edges, and the distance on the y-axis the final edge.

The hypotenuse shows us the line we want the box to travel along. But we don't want to travel there all in one go, we want to move along it slowly. If we take the triangle we have formed, and scale it down the angle of the hypotenuse is still the same, meaning if we keep moving in that direction we will eventually reach the end point, but the distance on the x and y axis are much smaller.

These smaller x and y distances (or deltas) are what we want to add to our position instead of 1.

If we were trying to animate from A to B, the red line shows how you would move if you always added 1. The dotted lines show the scaled triangles.

Math.sqrt(Math.pow(xToTarget, 2) + Math.pow(yToTarget, 2)) calculates the length of the hypotenuse in our triangle from current position to the target. We then use speed/length to create the scale we multiply the x and y lengths by. In this way the box will always move the equivalent of speed pixels diagonally towards the target.

I chose to keep track of the x and y position manually instead of reading it back from the dom element because I think it makes the code cleaner, and I think it should be faster.


Palindrome-testing Java program for an interview

It's enough to loop until word.length() / 2, as this will compare the first half with the second half, so no need to go until the end.

As you use word.length() multiple times, you could extract it to a helper variable.

There is a typo in the method name.

As for testing, yes, JUnit is the way to go. In a cloud IDE, you could create a helper method that takes a single string, calls isPalindrome and prints the result. That way you can test multiple cases easily, by adding one line per case. It's important to try to cover corner cases and potentially interesting cases, not just the "happy path". For example:

palindrome with even length
palindrome with odd length
single letter
empty string
non-palindromes

During an interview, it might also be worth mentioning the trade-off between comparing characters using .charAt, or using the array of characters returned by .toCharArray.


To answer your questions:

Yes if you never want to know why something went wrong. You should log the exception so you can diagnose issues later. Swallowing exceptions like that is rarely a good idea.
No. Web requests return status codes. It could be returning a 5xx error which would indicate that the directory was not created successfully but you'd be acting like it had worked. A response successfully is not the same thing as a successful response.


Smallest prime factor of a large number (using BigInteger)

Factors to consider

If number 2 is not a factor, then all large even numbers will not be factors. Therefore, checking only the odd numbers is sufficient.
For integers i x j = n, if i is larger than the square root of n, then j has to be smaller than the square root of n (Therefore, already checked). Hence, numbers higher than the square root of n do not have to be checked as candidates for prime factors.

private static BigInteger smallestPrimeFactor(BigInteger n) {
    BigInteger two = new BigInteger("2");
    if (n.mod(two).compareTo(BigInteger.ZERO) == 0) {
        return two;
    }

    for (BigInteger i = new BigInteger("3"); n.compareTo(i.multiply(i)) >= 0; i = i.add(two)) {
        if (n.mod(i).compareTo(BigInteger.ZERO) == 0) {
            return i;
        }
    }
    return n;
}


Is there an easier way to find the index of the first letter in a string?

I have no issue with the loop. It is about as fast as it can be. I would recommend renaming the method to match the other IndexOf methods.... something like IndexOfAlphaChar.

There is no reason to worry about this code. Move on to bigger problems.


Basic Nim game vs computer in JavaScript

I stumbled across this because a friend told me he got hit with this question recently for a coding bootcamp entrance exam. I had to look and see what it is, of course : )

Josh has a great answer with excellent use of higher order functions and keeping things DRY. At a glance, one change that I'd make with his code is the usage of the reduce function. It's a bit verbose and possibly difficult for a beginner to parse through. A clearer declaration would be using something like Array.every (once all the circles return an empty string "", the GameOver function returns true). It's a tiny gripe, but it's more intuitive. The usage would be substituted like this:

function isGameOver() {
  return piles.every(function(pile) {
    return pile.circles === "";
  });
}


Getting path array for given value in nested Object

Some thoughts:

This is not truly a recursive function, in that you are not passing the path value when first calling the function.  This may hint that you should really just recurse privately within the function itself such that you can better control "state" (mainly that of path and whether or not match has been found) and simplify things.
You can design away pretty much all of the temporary variables you use within the loop.
Why are you making a check for hasOwnProperty?  It seems like perhaps you are trying to exclude Arrays, Numbers, etc.  here? If that is the case, just check for the object's constructor rather than typeof up front and get rid of this.
Consider throwing on case where non-object is encountered in input.
Why mix return types?  If you are expecting an array as a result, why not have empty array represent "no result" case?

Putting it all together might yield something like:

function getPath(obj, value) {
    if(obj.constructor !== Object) {
        throw new TypeError('getPath() can only operate on object with Object as constructor');
    }
    var path = [];
    var found = false;

    function search(haystack) {
        for (var key in haystack) {
            path.push(key);
            if(haystack[key] === value) {
                found = true;
                break;
            }
            if(haystack[key].constructor === Object) {
                search(haystack[key]);
                if(found) break;
            }
            path.pop();
        }
    }

    search(obj);
    return path;
    /*
    Or alternately if you want to keep mixed return
    return found ? path : false;
    */
}

var nestedObj = {
  "rrr": {
    "ddd": {
      "aa    bbbb": {
        1: 30009463
      },
      "cc    dddd.": {
        1: 30010338
      },
      "e     fff": {
        1: 30007744
      },
      "d-Pool": {
        1: 30018363,
        2: 30017133,
        3: 30013107
      },
      "e   e g": {
        1: 30011707,
        2: 30017137
      },
      "f   f.-f. (f)": {
        1: 30012329
      },
      "g": {
        1: 30011894
      }
    }
  }
};

console.log(getPath(nestedObj, 30017137));
console.log(getPath(nestedObj, 'no value'));


Finding average of eight immediate neighbors of a matrix

Welcome to CodeReview!

Class naming

Maybe it's a term I'm not familiar with but Matrixer does not seem to ring a bell. Is it supposed to identify a "creator of matrixes"? Is it the name of your project? If you haven't made a conscious decision for this name, I would suggest turning it into something "universally" descriptive like MatrixCalculator (pretty much anything that contains the essence of what your class is supposed to achieve).

Indentation

Several places are indented incorrectly/inconsistently. Use a tab (4 spaces) for each new block level.

I would also layout code like your cellNeighborsAverage like this:

double sum = matrix[row - 1][col - 1] + matrix[row - 1][col]
           + matrix[row - 1][col + 1] + matrix[row][col - 1]
           + matrix[row][col + 1] + matrix[row + 1][col - 1]
           + matrix[row + 1][col] + matrix[row + 1][col + 1];

It's just a bit easier to read.

Likewise for your main method: the indentation is all over the place; maybe something went wrong when copy-pasting it here? Keep your if statements in this format:

if(condition) {
    doSomething();
} elseif (anotherCondition) {
    doAnotherThing();
} else {
    doNothing();
}

Parameter naming

Your constructor takes variables N and M in a scientific-oriented context this is an acceptable practice so it will depend on how you view this code but standard conventions state that variables use the lowerCamelCase convention.

I would also make their meaning clearer: N could become rows and M would be columns.

Likewise your parameter is named imatrix. What does that i signify? You can simply call it matrix and use this.matrix to refer to the instance variable inside the constructor body.

I would also distinguish between matrix and computedMatrix. Computed what exactly? Maybe sourceMatrix and averagedMatrix are more appropriate terms?

Method naming

cellNeighborsAverage doesn't really tell me much about what goes on in there. Something about the neighbours average, that's for sure. But what exactly does it do? Does it calculate it? Return it? Do I have to set it?

Conventions state that your method names are built up like [verb][subject] which would become getAverageFromNeighbours.

NullPointerException

The NPE is something you should always prevent and guard against instead of throwing/catching it at runtime. You know the flow of your program and thus you should be able to determine where a NPE could occur. Guard yourself against this by validating parameters, instantiating fields inline when you can, avoid returning null, etc.

Scanners

You only need one Scanner on your System.in which you re-use to get input.

Returning instead of printing

Right now you're working with void methods that print the information directly to the console. This will bring problems when you decide to publish your library (class) online and let people use it. What if that person now wants to write the results to a textfile instead of the console?

Likewise for unit testing this will become troublesome.

Therefore I suggest that you return the results from your methods instead of printing to the console inside Matrixer itself; let the main class decide what happens with your results.


Syntax errors aside, there's a major pitfall in your design choices that you need to watch out for. Namely, if you want to figure out if your animal is capable of some behaviour, you'll need to add an identifier function for every animal. This will limit you down the road.

For example, say you are iterating over a bunch of different types of animals, you'd need something like:

foreach ($noahs_ark->animals() as $animal) {
    if ($animal->canFly()) {
        $animal->performFly();
    }
}

The problem here is that if you want to be able to check that an animal can fly, every single animal will need a "canFly()" method otherwise you'll end up with an error. If you're dealing only with birds, this probably isn't a big deal. But when you add dogs to the mix, then you have to add a ->canFly() method for every dog object too!

The solution to this is to implement your interfaces directly on your animal classes. For example:

// Assuming all birds classes extend Avian
abstract class Avian implements Flier, Quacker { ... }

foreach ($noahs_ark->animals() as $animal) {
    if ($animal instanceof Flier) {
        $animal->performFly();
    }
]

This way you can use reflection to determine types, and along with it, make assumptions that if it possesses(implements) a certain behaviour(interface), your animal can perform that action.

I'd still recommend that you make your modules implement interfaces as well to ensure consistency across specialized modules though. It'll help in cases where you might want to create factories for creating animals or in other cases as well. Hope that helps!


There are some minor issues with this:

The condition to check the value of choice is duplicated
The program will crash if the input is empty
The variable choice is declared outside the loop body
The formatting is a bit unusual

I suggest this way instead:

while (true) {
    System.out.println("Would you like to continue?(y/n)");
    String line = sc.nextLine();
    if (!line.isEmpty()) {
        char choice = line.charAt(0);
        if (choice == 'y' || choice == 'n') {
            return choice;
        }
    }
    System.out.println("Invalid Entry");
}


Constructing query conditions from answers that may be null

Since you are doing the same operation over and over, you can put your strings in an array or list and use a mapping function.

For example:

//this struct makes it easy to initialize the list of values
public struct Tuple
{
  public String Value;
  public String InterpolatedValue;
  public Tuple(string val, string def)
  {
    Value = val;
    InterpolatedValue = def;
  }
}

//dummy answers to the questions
string question2Answer = "aaa";
string question3Answer = "bbb";
string question4Answer = "";
string question5Answer = "ddd";

//initialize a list of answer/interpolated answer strings
List<Tuple> list = new List<Tuple>()
{
  new Tuple(question2Answer, $" and ProdUsers = {question2Answer}"),
  new Tuple(question3Answer, $" and ProdFullyOnline = '{question3Answer}'"),
  new Tuple(question4Answer, $" and ProdMultiCurrency = '{question4Answer}'"),
  new Tuple(question5Answer, $" and ProdPrice <= {question5Answer}"),
};

//loop over list, checking for non empty answer
//this line and the next takes the place of almost all of your original code
IEnumerable<string> tempList= list.Select(t => (string.IsNullOrWhiteSpace(t.Value) ? string.Empty : (t.DefaultValue)));

//concat results into a single string
string text = string.Join("", tempList);

Console.WriteLine(text);

Outputs: " and ProdUsers = aaa and ProdFullyOnline = 'bbb' and ProdPrice <= ddd"

I had to add some boilerplate code to make the example work, but I think one gets the idea...


Reverse each word in the string. All spaces in the string should be retained

The split function

vector<string> split(string s, string delimiter)
{
    size_t pos_start = 0, pos_end, delim_len = delimiter.length();
    string token;
    vector<string> res;

    while ((pos_end = s.find(delimiter, pos_start)) != string::npos)
    {
        token = s.substr(pos_start, pos_end - pos_start);
        pos_start = pos_end + delim_len;
        res.push_back(token);
    }

    res.push_back(s.substr(pos_start));
    return res;
}

delim_len should be declared as const, because it doesn't change.  Or simply drop it; it is only used in one place.

This function forces all substrings to be first copied into the token variable and then copied into the vector — that's two copies.  It can be reduced to one copy by using std::move (defined in <utility>) on the argument to push_back, or by simply dropping the token variable.

The function should have std::string_view (defined in <string_view>) parameters instead of std::string parameters for efficiency.

Putting everything together:

auto split(std::string_view s, std::string_view delimiter)
{
    std::vector<std::string> result;

    std::size_t pos_start = 0, pos_end;
    while ((pos_end = s.find(delimiter, pos_start)) != s.npos) {
        res.push_back(s.substr(pos_start, pos_end - pos_start));
        pos_start = pos_end + delimiter.size();
    }

    res.push_back(s.substr(pos_start));
    return res;
}

vector_to_string

string vector_to_string(vector<string> v)
{
    string s;

    for (auto &e : v)
    {
        reverse(e.begin(), e.end());
        s += e + " ";
    }

    return s;
}

vector_to_string is a non-descriptive name — reverse_join may be better.  The delimiter should be an argument rather than hard coded.

This function should strip the last space instead of letting its caller do so.  Note that this function performs two operations logically (reverse and join).

Copying the whole vector takes a lot of space; consider copying one string at a time.

And yes, there is a simpler way to do this using std::accumulate (defined in <numeric>):

auto result = std::accumulate(v.begin(), v.end(), std::string{},
                              [](std::string lhs, std::string_view rhs) {
                                  lhs += rhs;
                                  lhs += ' ';
                                  return lhs;
                              });
result.pop_back();

reverse_words

string reverse_words(string str)
{
    vector<string> v = split(str, " ");
    string s = vector_to_string(v);
    return s.substr(0, s.length() - 1);
}

Again, std::string_view parameter.  Even if the stripping happens here, it should be done using pop_back to avoid copying the string and enable named return value optimization.

Simplification

Here's a simplification, using streams to do the split job:

#include <cassert>
#include <iostream>
#include <sstream>
#include <string>
#include <string_view>

std::streamsize read_spaces(std::istream& is)
{
    std::streamsize count = 0;
    while (is.peek() == ' ') {
        is.get();
        ++count;
    }
    return count;
}

// const std::string& because std::istringstream is constructed from a std::string
std::string reverse_words(const std::string& string)
{
    std::istringstream iss{string};
    std::ostringstream oss{};

    for (std::string word; iss >> word;) {
        std::reverse(word.begin(), word.end());
        oss << word << std::string(read_spaces(iss), ' ');
    }
    return oss.str();
}

int main()
{
    assert(reverse_words("The quick brown fox jumps over the lazy dog.") == "ehT kciuq nworb xof spmuj revo eht yzal .god");
    assert(reverse_words("apple") == "elppa");
    assert(reverse_words("a b c d") == "a b c d");
    assert(reverse_words("double  spaced  words") == "elbuod  decaps  sdrow");
    assert(reverse_words("") == "");
    assert(reverse_words("ab   ba   cd") == "ba   ab   dc");
}

(live demo)

Miscellaneous

#include <string>
#include <iostream>
#include <vector>
#include <algorithm>

Sort the #includes in alphabetical order to each navigation.

using namespace std;

Avoid global using namespace std;.  It brings in many common identifiers from the std namespace and causes name clashes.  Although it doesn't matter too much for a small program, get into the habit of explicitly qualifying names with std:: instead.  An alternative is using declarations (using std::string, std::vector;), which only pulls in the specified names.  See Why is using namespace std; considered bad practice? for more information.


Removal of a character to make a string a palindrome

Some Improvements:

Your checkPalindrome function is returning whether the string is a palindrome or not. What if it returns an index, at which it fails?
Try working with that. Right now, you are calling the checkPalindrome function O(string-length), and checkPalindrome in itself is O(string-length), making it a square time algorithm.

Wikipedia article on Big O notation
A minor improvement: do you need to iterate over the entire string to determine if it is a palindrome?


Tracking stock prices using Python and Google Sheets

This looks very interesting. Well done!

Here is some nitpicking:

from yahoofinancials import YahooFinancials
import re
import pygsheets
import pandas as pd
import datetime

Organise these imports in following way: builtins, third-party then first-party.

 def main(sheet, everything,place):

Name of this function is sloppy. Use main function as an entry-point.
Name the function as a verb phrase or that explains what it is doing.
What is everything here? I can see that it is defined outside as well.
Avoid using everything as a name. It is very vague.

for i in tickers:
    tick = YahooFinancials(i)

What is i here? Reserve i for integer indexes. These are ticker_names.
for ticker_name in tickers is more readable.

tickers = ['XOM', 'JNJ', 'TR', 'CRON', 'HSY', 'FL', 'PLNT', 'MCD', 'ARLP', 'LULU', 'RCII', 'DELL', 'DNKN', 'DIS']

Load this from a JSON file.

def get_name(person):

This function doesn't get you anything. There is no return.
Why does this call main?
Name to sheet mapping can probably be loaded from a JSON as well.

adjclose, PY to Gsheet Test

Perform extract constant refactoring on strings like these and remove magic values.

  A magic number is a direct usage of a number in the code.

  For example, if you have (in Java):

  public class Foo {
    public void setPassword(String password) {
         // don't do this
         if (password.length() > 7) {
              throw new InvalidArgumentException("password");
         }
    }
}

  This should be refactored to:

  public class Foo {
    public static final int MAX_PASSWORD_SIZE = 7;

    public void setPassword(String password) {
         if (password.length() > MAX_PASSWORD_SIZE) {
              throw new InvalidArgumentException("password");
         }
    }
}

  From: https://stackoverflow.com/questions/47882/what-is-a-magic-number-and-why-is-it-bad

Summary - This code is very hard to understand. Functions do things that are completely different from function-name. Variable names are vague. Contains lot of hard-coded values that can be extracted to a JSON or some other kind of config. Contains magic values that should be converted to meaningful constants.


Using view for repeated subquery on a large table

Since your primary key contains both src and dst you might be able to get away with something like the following:

SELECT IF (src<dst,src,dst), COUNT(*)
FROM   iplogs
GROUP BY 1
UNION
SELECT IF (src<dst,dst,src), COUNT(*)
FROM iplogs
GROUP by 1

An explain suggests the query above can use the primary index (assuming the captured field is not first or that a suitable expression using the captured date is added to the where clause if it is first).

#id, select_type, table,  type,  possible_keys, key,     key_len, ref, rows, Extra
1,   PRIMARY,     iplogs, index, PRIMARY,       PRIMARY, 105,        , 6,    Using index; Using temporary; Using filesort
2,   UNION,       iplogs, index, PRIMARY,       PRIMARY, 105,        , 6,    Using index; Using temporary; Using filesort
 ,   UNION RESULT, <union1,2>,   ALL, , , , , , Using temporary

If you put EXPLAIN in front of your query you'll get details on how it will execute.

Getting down to two index scans should be as efficient as you can get and a bit simpler than using a view.

However, beware, I've only done rudimentary testing on a small data set at this point.

EDIT: With more details provided I've been able to test this and provide an improved response.

The union query given below returns two distinct portions of results. Adding these two together appears to provide the proper total.  Here is a bit of PHP/MySQL code taken from some testing efforts against the supplied data.

$sql  = "SELECT IF (src<dst,src,dst) src\n";
$sql .= "      ,COUNT(DISTINCT(IF (src<dst,concat(src,'|',dst),concat(dst,'|',src)))) answer\n";
$sql .= "      ,COUNT(*) raw\n";
$sql .= "FROM   iplogs\n";
$sql .= "GROUP BY 1\n";
$sql .= "UNION\n";
$sql .= "SELECT IF (src<dst,dst,src)\n";
$sql .= "      ,COUNT(DISTINCT(IF (src<dst,concat(src,'|',dst),concat(dst,'|',src))))\n";
$sql .= "      ,COUNT(*)\n";
$sql .= "FROM iplogs\n";
$sql .= "GROUP by 1\n";
if ( !$res = mysqli_query($link,$sql) )
  die("ERROR: Unable to query vertex counts from database!\n");

However, as you can imagine, you have to post-process the query in order to add the values together to get what should be the right answer.  This could be done by loading the returned records into a vertex table as is if consuming processes group by IP and sum the response.

Alternately, here is some code that does the summing and would produce records suitable for storing in a vertex table.

$qryqty = 0;
$qrylist = array();
while ( $row = mysqli_fetch_assoc($res) )
{
  $src = $row['src'];
  if ( !isset($qrylist[$src]) )
    $qrylist[$src] = 0;
  $qrylist[$src] += $row['answer'];

  $qryqty++;
}
mysqli_free_result($res);
echo "$qryqty records loaded via query<br>\n";

So, I guess there is a trade off between SQL efficiency and having to do some extra processing to sum the results either before or after storing them.

NOTE: There are no IP translations as they were loaded as text for this effort.


Display the difference between DataFrames' dtypes?

self

Why put this method on a class? the lack of use of self in the method should act as a flag

string result

you format the mismatch as a string (f"df1:{val}, df2: {df2.dtypes[key]}"). This way you can not do anything about it any more. Better would be to use a tuple or dict here

generator

generally, when I see

def method():
    result = {}
    for key in ...:
        result[key] = ...
    return key

it is more clear to work with a generator

def method():
    for key in ...:
        yield key, ...

If the user of the method needs a dict, he can do dict(method()). If he wants to iterate over the items, there is no need to instantiate the dict

alternative approach

you can use sets to see which types are different:

def get_mismatched_dtypes(df1, df2):
    df1_types = set(df1.dtypes.items())
    df2_types = set(df2.dtypes.items())
    for column_name, df1_type in df1_types - df2_types:
        yield column_name, (df1_type, df2.dtypes[column_name])

df1 = pd.DataFrame({"col1": [1, 2, 3], "col2": [4, 5, 6]}, index=list(range(3)))
df2 = pd.DataFrame({"col1": list("abc"), "col2": [4, 5, 6]}, index=list(range(3,6)))
dict(get_mismatched_dtypes(df1, df2))

{'col1': (dtype('int64'), dtype('O'))}

for clarification: df1_types - df2_types is {('col1', dtype('int64'))}

If you want it formatted like in your code:

{
    column_name: f"df1: {df1_type}, df2: {df2_type}"
    for column_name, (df1_type, df2_type) in get_mismatched_dtypes(df1, df2)
}

{'col1': 'df1: int64, df2: object'}


It's OK to use the if construction, although if you plan to have more than a few commands, using switch leads to cleaner code imho. In addition, why not using if else instead of just if? That way not all of your if have to be evaluated (in average).


Title capitalization exercise from The Odin Project

Your code looks good. But I have some suggestions. lowercase array will be created on each method call, so we can just declare it as class level constant (and use %w() syntax):

LOWERCASE = %w(and or the of in a an).freeze

Try to use single quotes if possible.

It is possible to use next inside map block to avoid if/else. And split(' ') is the same as split.

Regarding specs. There's neat RSpec feature called shared_examples(RSpec shared_example).

Here's my suggested refactoring:

class Book
  attr_accessor :title

  LOWERCASE = %w(and or the of in a an).freeze

  def title=(book)
    @title = book.capitalize.split.map do |word|
      next word if LOWERCASE.include?(word)
      word.capitalize
    end.join(' ')
  end

end

RSpec.shared_examples 'capitalize' do |message, text, title|
  let!(:book) { described_class.new }
  let!(:assign_title) { book.title = text }

  it message do
    expect(subject).to eq(title)
  end
end

RSpec.describe Book do
  context 'title' do
    subject { book.title }

    it_behaves_like 'capitalize', 'first letter', 'inferno', 'Inferno'
    it_behaves_like 'capitalize', 'every word', 'stuart little', 'Stuart Little'

    context 'articles' do
      it_behaves_like 'capitalize', 'except "the"', 'alexander the great', 'Alexander the Great'
      it_behaves_like 'capitalize', 'except "a"', 'to kill a mockingbird', 'To Kill a Mockingbird'
      it_behaves_like 'capitalize', 'except "an"', 'to eat an apple a day', 'To Eat an Apple a Day'
    end

    it_behaves_like 'capitalize', 'except conjunctions', 'war and peace', 'War and Peace'
    it_behaves_like 'capitalize', 'except prepositions', 'love in the time of cholera', 'Love in the Time of Cholera'
    it_behaves_like 'capitalize', 'always for "I"', 'what i wish i knew when i was 20', 'What I Wish I Knew When I Was 20'
    it_behaves_like 'capitalize', 'always for first word', 'the man in the iron mask', 'The Man in the Iron Mask'
  end
end


Producing the intersection of several sequences

First let me point out that you shouldn't name this extension method as you have named: Intersect(). The name Intersect() already exists and will return the intersection of two collections.  Your method is intersecting a collection of collections with each other.  It should must be named differently.  I would suggest IntersectAll() or possibly IntersectMany().

I feel your implementation for the case of a single collection is wrong.  You are performing a set operation here.  You need to ensure that for all cases, you maintain set semantics.  You assume that for a single collection that you may return that collection.  But what if that collection contains duplicates?  You would need to filter out those duplicates.  The aggregated version should as well.

With that in mind, I would change your implementation like so:

public static IEnumerable<TSource> IntersectAll<TSource>(
        this IEnumerable<IEnumerable<TSource>> source)
{
    using (var enumerator = source.GetEnumerator())
    {
        if (!enumerator.MoveNext())
            yield break;

        var set = new HashSet<TSource>(enumerator.Current);
        while (enumerator.MoveNext())
            set.IntersectWith(enumerator.Current);
        foreach (var item in set)
            yield return item;
    }
}

I'm not sure how you did your testing but even with your implementation, I still get much faster results, even in the single collection case.  Tested in LINQPad with optimizations enabled:

Your implementation:

public static IEnumerable<TSource> IntersectAll1<TSource>(this IEnumerable<IEnumerable<TSource>> source)
{
    using (IEnumerator<IEnumerable<TSource>> sourceIterator = source.GetEnumerator())
    {
        //if there is no first element return an empty list
        if (!sourceIterator.MoveNext())
        {
            yield break;
        }

        IEnumerable<TSource> firstSequence = sourceIterator.Current;
        if (sourceIterator.MoveNext())
        {
            //create a hashset of the first sequence
            HashSet<TSource> hashSet = new HashSet<TSource>(firstSequence);

            //intersect the other sequences
            do
            {
                hashSet.IntersectWith(sourceIterator.Current);
            }
            while (sourceIterator.MoveNext());
            foreach (TSource element in hashSet)
            {
                yield return element;
            }
        }
        else
        {
            //if there is only one sequence, return it without any intersection
            foreach (TSource element in firstSequence)
            {
                yield return element;
            }
        }
    }
}

public static IEnumerable<TSource> IntersectAll2<TSource>(
        this IEnumerable<IEnumerable<TSource>> source)
{
    using (var enumerator = source.GetEnumerator())
    {
        if (!enumerator.MoveNext())
            yield break;

        var set = new HashSet<TSource>(enumerator.Current);
        while (enumerator.MoveNext())
            set.IntersectWith(enumerator.Current);
        foreach (var item in set)
            yield return item;
    }
}

public static IEnumerable<TSource> IntersectAllControl<TSource>(
        this IEnumerable<IEnumerable<TSource>> source)
{
    return source.Aggregate((x, y) => x.Intersect(y));
}

void Main()
{
    var rng = new Random();
    var collections = Enumerable.Range(0, 500)
        .Select(i => Enumerable.Range(rng.Next(0, 50), 100).ToArray())
        .Cast<IEnumerable<int>>()
        //.Take(1)
        .ToArray();

    const int Iterations = 1000; // 1000000 for single collection

    object result1 = null;
    var timer1 = Stopwatch.StartNew();
    for (var i = 0; i < Iterations; i++)
        result1 = collections.IntersectAll1().ToList();
    timer1.Stop();
    result1.Dump(String.Format("result1: {0}", timer1.Elapsed));

    object result2 = null;
    var timer2 = Stopwatch.StartNew();
    for (var i = 0; i < Iterations; i++)
        result2 = collections.IntersectAll2().ToList();
    timer2.Stop();
    result2.Dump(String.Format("result2: {0}", timer2.Elapsed));

    object controlResult = null;
    var controlTimer = Stopwatch.StartNew();
    for (var i = 0; i < Iterations; i++)
        controlResult = collections.IntersectAllControl().ToList();
    controlTimer.Stop();
    controlResult.Dump(String.Format("controlResult: {0}", controlTimer.Elapsed));
}

Note: The ToList() calls were needed on all invocations to ensure that the intersections are actually generated.

Run on my machine with multiple collections:

result1:       00:00:01.0210291
result2:       00:00:01.0285069
controlResult: 00:00:03.1512838

And with a single collection:

result1:       00:00:02.9254441
result2:       00:00:04.8505489
controlResult: 00:00:00.2102433

I have to apologize, I have changed my review as I wrote this due to some details I overlooked as I was writing. Thank you mjolka for pointing out my mistake.

For multiple collections, the new implementations will always be a win over the aggregation.  In the aggregated approach, each sub-instersection needs to be determined for every collection.  In our new implementations, only a single set intersection needs to be maintained.

The new implementations will not beat out the aggregate method in the single collection case however, because the aggregate will simply pass the collection through as the result untouched.  The new implementations will always do some bit of processing.  However again, I feel that if set semantics were preserved, this difference will probably be cancelled out anyway.