Scraping data unveiling a button from craigslist

There are some things I would do differently:

create a class - this way you may share a web-scraping session between the methods and also share the base and start urls
use more meaningful variable and method names - for instance, parse_doc() could be get_contact_info(); you can re-use page variable names in different methods etc.
you can use findtext() method to get the text
I would also return from the scraper and print out the results outside of it
you are searching for the phone number twice - instead, use the .search() method and check if you've got a match object or none
you can pre-compile and re-use the regular expression pattern for the phone number

Improved code:

import re

import requests
from lxml import html

class CraigListScraper:
    PHONE_NUMBER_PATTERN = re.compile(r'\d{10}')

    def __init__(self, start_url, base_url):
        self.session = requests.Session()
        self.base_url = base_url
        self.start_url = start_url

    def scrape(self):
        page = self.session.get(self.start_url).text

        tree = html.fromstring(page)
        for row in tree.xpath('.//li[@class="result-row"]'):
            link = self.base_url + row.xpath(".//a[contains(concat(' ', @class, ' '), ' hdrlnk ')]/@href")[0]

            yield self.process_search_result(link)

    def process_search_result(self, medium_link):
        page = self.session.get(medium_link).text
        tree = html.fromstring(page)

        name = tree.findtext('.//span[@id="titletextonly"]')

        try:
            contact_info_link = self.base_url + tree.xpath('//section[@id="postingbody"]//a[@class="showcontact"]/@href')[0]
            phone_number = self.get_contact_info(contact_info_link)
        except IndexError:
            phone_number = ""

        return name, phone_number

    def get_contact_info(self, target_link):
        page = self.session.get(target_link).text

        match = self.PHONE_NUMBER_PATTERN.search(page)
        return match.group(0) if match else ""

if __name__ == '__main__':
    start_url = "http://bangalore.craigslist.co.in/search/reb?s=120"
    base_url = "http://bangalore.craigslist.co.in"

    scraper = CraigListScraper(start_url, base_url)
    for result in scraper.scrape():
        print(result)


Yes, I think it is better to use efficient base functions for this kind of task. I'm assuming your ids are sorted so that duplicates will always be next to each other, is that right? In that case, a first idea would be to use the duplicated function. Using your example, see that:

x <- c(1, 1, 2, 2, 3, 4)
duplicated(x)
# [1] FALSE  TRUE FALSE  TRUE FALSE FALSE
1 + duplicated(x)
# [1] 1 2 1 2 1 1

However, this will not work in your more general case where there may be more than 2 duplicated households:

x <- c(1, 1, 1, 2, 2, 2, 3, 3, 4, 5)
1 + duplicated(x)
# [1] 1 2 2 1 2 2 1 2 1 1

For the general case, I would use run length encoding functions:

rle(x)
# Run Length Encoding
#   lengths: int [1:5] 3 3 2 1 1
#   values : num [1:5] 1 2 3 4 5
sequence(rle(x)$lengths)
# [1] 1 2 3 1 2 3 1 2 1 1

So I would suggest you replace your assign_extra_id with the following:

assign_extra_id2 <- function(x) sequence(rle(x)$lengths)

Update

Taking into account the added details (in the comments and updated question), maybe this function will do?

assign_extra_id3 <- function(x) {
   resample <- function(x, ...) x[sample.int(length(x), ...)]
   z <- rle(rle(x)$lengths)
   s <- sample(z$values[1])
   i <- Map(function(l,v) rep(resample(s[1:v]),l), z$lengths, z$values)
   unlist(i, use.names = FALSE)
}

It's (still) a nice speedup:

# Unit: milliseconds
#        expr        min         lq       mean    median         uq       max neval
 # op_attempt 5013.47742 5051.44837 6198.35973 5535.9724 6951.75439 9843.3606    10
 # fd_attempt   54.48751   59.92706   73.31207   62.1226   82.82826  130.2046    10


Checking return values

A mistake I see in multiple places is not checking the return values of operations.

Take for example the part of getting the user ID:

PreparedStatement prepStmt = conn.prepareStatement("SELECT userId FROM SiteUser WHERE name =?");
prepStmt.setString(1, username);
ResultSet rs = prepStmt.executeQuery();
rs.next();
return rs.getInt(1);

Before getting values from a ResultSet instance,
you should always check the return value of rs.next(),
as explained in the official JDBC tutorial.

Without that check, the rs.getInt(1) statement will raise SQLException when there is no matching user.
Since you wrapped the block of code in try { ... } catch (SQLException e) { ... }, the exception will be handled, but this is not appropriate use of exceptions.

The purpose of the try-catch is not to handle the case of no matching user.
Its purpose is to handle unexpected errors during the database operation.
If you rely on this try-catch to handle the case of no matching user,
that's effectively using exceptions for flow control.
That's not the intended use, and it's also inefficient,
as it's slower than using a conditional to check the value of rs.next().

I suggest to review the entire code, look for statements where a function returns a value but you don't use it. Some IDEs for example IntelliJ will give you a warning on statements like that.

Another form of not checking the return value is in the process of registering a user. Consider this snippet from the register method:

insertNewUser(username, pass);
createAccountForUser(username);

The insertNewUser doesn't return anything,
but probably it should.
Inserting a new user might not succeed,
and it would be incorrect to create an account if the user could not be created.

Potentially unsafe register

The implementation looks prone to a race condition:
in between the moment of checking if the username exists,
and the moment of inserting the user,
another process or program may have inserted a user with the same name.
In fact,
unless you wrap these two operations within a transaction,
there's no way to safely guarantee that the insert operation will succeed.

I'm not sure if you have a unique index on the username column.
If you do, then you could skip checking if the user exists,
attempt to insert.
If the insert is successful, the user did not exist, it was successfully inserted, and you can continue creating the account.
If the insert fails, whether because the user already exists, or due to other database errors, you can abort further processing.

Connection management

Creating connections is an expensive operation.
Avoid creating new connections unnecessarily as much as possible.
It might be a good idea to manage the connection outside this class.
Or if you really want to create it in this class,
make sure that a chain of operations reuses the same connection,
rather than recreating it in each step.

Ordering of operations

The register method first checks if the specified username exists,
and then validates the username and the password against some built-in rules.
It would be better to perform these operations in the opposite order.
The validation logic is a cheap operation,
database lookups are expensive.

Confusing member and local variables

Some methods reuse the conn field,
others create it as a local variable.
This is confusing.


Better way to write nested loop, if condition and breaking it

My implementation:

(Is C#.Net but you'll be able to translate easily).

    public class Point()
    {
        public int X {get; set;}
        public int Y {get; set;}

        public Point (int x, int y)
        {
            this.X=x;
            this.Y=y;
        }
    }

    public void calcLine(bool[][] grid)
    {
        int rows = 500;
        int columns = 500;

        Point p1 = new Point(0, 0);
        // Change this outer 'if' to 'while' if should continue searching for all lines to draw
        if (canFindNextPoint(grid, rows, columns, p1))
        {
            Point p2 = new Point(p1.X + 1, 0);
            if (canFindNextPoint(grid, rows, columns, p2))
            {
                drawline(p1.X, p1.Y, p2.Y, p2.Y);
            }
        }
    }

    private bool canFindNextPoint(bool[][] grid, int rows, int columns, Point p)
    {
        bool found = false;
        for (int x = p.X; x < rows; x++)
            for (int y = p.Y; y < columns; y++)
            if (grid[x][y])
            {
                found = true;
                p.X = x;
                p.Y = y;
                break;
            }
        return found;
    }


Test four int of commonality in the lower four bits

The code isn't bad, but it's a little more verbose than it needs to be.  Consider that we don't really need to check one bit at a time; we can check four simultaneously.

The key here is that we're looking for bits that are the same in all four numbers.  If we wanted to look for ones that were shared, we could do this:

a & b & c & d & 0xf

If we want to look for zeroes, we can simply invert:

~a & ~b & ~c & ~d & 0xf

If we put those together with the -1 part, it might look like this:

return a != -1 && b != -1 && c != -1 && d != -1 &&
    ((a & b & c & d) || (~a & ~b & ~c & ~d & 0xf));

However, the problem with this is that even though it's parallel, it still requires more operations than might be required.

If we consider the exclusive or function, it effectively return a 1 whenever the bits differ.  So if there were only two numbers, we could do:

return (a ^ b) ^ 0xf;

The expression would only be false if all of the bits were different.  We can use a similar strategy for three numbers.

return ((a ^ b) | (b ^ c)) ^ 0xf;

For this one, (a ^ b) returns 1 for every bit that is not the same between a and b, and the expression (b ^ c) returns 1 for every bit that is not the same between b and c.  When we do a bitwise or of those quantities, only the the bits that are the same for all three quanties remain zeroes.  When we do ^ 0xf we invert the bottom 4 bits so that only bits that are the same are ones.

Extrapolating this to four quantities,

return ((a ^ b) | (b ^ c) | (c ^ d)) ^ 0xf;

This is good but not sufficient since we still need to deal with the possible -1 quantities that may be among the inputs.  One obvious way to do this is this:

return a != -1 && b != -1 && c != -1 && d != -1 &&
    (((a ^ b) | (b ^ c) | (c ^ d)) ^ 0xf);

To compare this routine which I'll call "Edward" to the one above, which I'll call "naive" to the original and the other three proposals so far, I used this code:

testcode.c

#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <math.h>
#include <stdbool.h>

bool sim(int a, int b, int c, int d)
{
    return ((a != -1) && (b != -1) && (c != -1) && (d != -1)) &&
    ((((a & 8) == (b & 8)) && ((a & 8) == (c & 8)) && ((a & 8) == (d & 8))) ||
     (((a & 4) == (b & 4)) && ((a & 4) == (c & 4)) && ((a & 4) == (d & 4))) ||
     (((a & 2) == (b & 2)) && ((a & 2) == (c & 2)) && ((a & 2) == (d & 2))) ||
     (((a & 1) == (b & 1)) && ((a & 1) == (c & 1)) && ((a & 1) == (d & 1))));
}

bool naive(int a, int b, int c, int d)
{
    return a != -1 && b != -1 && c != -1 && d != -1 &&
        ((a & b & c & d) || (~a & ~b & ~c & ~d & 0xf));
}

bool edward(int a, int b, int c, int d)
{
    return a != -1 && b != -1 && c != -1 && d != -1 &&
        (((a ^ b) | (b ^ c) | (c ^ d)) ^ 0xf);
}

bool BitsInCommon(int a, int b, int c, int d)
{
    if (a == -1 || b == -1 || c == -1 || d == -1)
        return false;

    int rslt = 0;
    rslt = (a & 15) & (b & 15) & (c & 15) & (d & 15);
    if (rslt != 0)
        return true;

    //check for zero bits in common???  if needed?
    rslt = ((a ^ -1) & 15) & ((b ^ -1) & 15) & ((c ^ -1) & 15) & ((d ^ -1) & 15);

    return rslt != 0;
}

bool jan_korous(int a, int b, int c, int d) {

    if( a == -1 || b == -1 || c == -1 || d == -1 ) { return 0; }

    const unsigned int diff = (a ^ b) | (c ^ d) | (a ^ c);

    return
        ( (diff & 1) == 0 ) ||
        ( (diff & 2) == 0 ) ||
        ( (diff & 4) == 0 ) ||
        ( (diff & 8) == 0 );

}

bool scottbb(int a, int b, int c, int d) {
    if (a == -1 || b == -1 || c == -1 || d == -1) {
        return 0;
    }
    else {
        unsigned int all_1 = (unsigned int)a & (unsigned int)b &
                             (unsigned int)c & (unsigned int)d;
        unsigned int all_0 = ~((unsigned int)a | (unsigned int)b |
                               (unsigned int)c | (unsigned int)d);
        return (all_1 | (all_0 & 0xF)) ? 1 : 0;
    }
}

int main()
{
    bool troubleshoot = false;
    struct {
        const char *name;
        bool (*func)(int, int, int, int);
        double elapsed;
        bool isCorrect;
    } tests[] = {
        { "original", sim, 0, true },
        { "naive", naive, 0, true },
        { "Edward", edward, 0, true },
        { "dbasnett", BitsInCommon, 0, true },
        { "Jan Korous", jan_korous, 0, true },
        { "scottbb", scottbb, 0, true},
        { NULL, NULL, 0, false}
    };
    // see if they're all correct
    for (int a = -1; a < 16; ++a) {
        for (int b = -1; b < 16; ++b) {
            for (int c = -1; c < 16; ++c) {
                for (int d = -1; d < 16; ++d) {
                    for (size_t i = 1; tests[i].func; ++i) {
                        if (tests[i].func(a,b,c,d) != tests[0].func(a,b,c,d)) {
                            if (troubleshoot) {
                                printf("%s failed! [%d, %d, %d, %d] => %d, should have been %d\n",
                                    tests[i].name, a, b, c, d,
                                    tests[i].func(a, b, c, d),
                                    tests[0].func(a, b, c, d)
                                );
                            }
                            tests[i].isCorrect = false;
                            if (troubleshoot)
                                return -1;
                        }
                    }
                }
            }
        }
    }

    puts("All functions checked for accuracy; checking timing...");
    for (int iterations = 100; iterations; --iterations) {
        for (size_t i = 0; tests[i].func; ++i) {
            if (tests[i].isCorrect) {
                for (int a = -1; a < 16; ++a) {
                    for (int b = -1; b < 16; ++b) {
                        for (int c = -1; c < 16; ++c) {
                            for (int d = -1; d < 16; ++d) {
                                clock_t start = clock();
                                tests[i].func(a,b,c,d);
                                tests[i].elapsed += clock() - start;
                            }
                        }
                    }
                }
            }
        }
    }
    // print results
    for (size_t i = 0; tests[i].func; ++i) {
        if (tests[i].isCorrect) {
            printf("%12s\t%.10f\t%f%% %s than %s\n", tests[i].name, tests[i].elapsed,
                100.0*fabs(tests[i].elapsed-tests[0].elapsed)/tests[0].elapsed,
                (tests[i].elapsed > tests[0].elapsed ? "slower" : "faster"),
                tests[0].name
            );
        } else {
            printf("%12s\twas not correct; no time recorded\n", tests[i].name);
        }
    }
}

Results:

Note: I've updated the results to include everybody's correct versions and to include the two solutions proposed by @js1 although I didn't bother duplicating the source code for those solutions in the code above.

All functions checked for accuracy; checking timing...
    original    1052389.0000000000  0.000000% faster than original
       naive    1034792.0000000000  1.672100% faster than original
      Edward    1024637.0000000000  2.637048% faster than original
    dbasnett    1032151.0000000000  1.923053% faster than original
  Jan Korous    1028349.0000000000  2.284326% faster than original
     scottbb    1026372.0000000000  2.472185% faster than original
         JS1    1035062.0000000000  1.646444% faster than original
  JS1 Lookup    1023049.0000000000  2.787942% faster than original

So in my tests, on a quad-core x86_64 machine, the Edward and Jan Korous routines are nearly identical in time and both are a small improvement (approximately 2%) over the original.  There was a slight but persistent speed advantage to the "JS1 Lookup" version on this machine.

Compile command (Linux) was:

  gcc -O2 -std=c99 testcode.c -o testcode && ./testcode

On a quad core ARM7, I got this result with the same code:

All functions checked for accuracy; checking timing...
    original    17302157.0000000000 0.000000% faster than original
       naive    17211229.0000000000 0.525530% faster than original
      Edward    17231692.0000000000 0.407261% faster than original
    dbasnett    17255411.0000000000 0.270174% faster than original
  Jan Korous    17115011.0000000000 1.081634% faster than original
     scottbb    17120390.0000000000 1.050545% faster than original
         JS1    17267606.0000000000 0.199692% faster than original
  JS1 Lookup    17118642.0000000000 1.060648% faster than original

I also tested on Windows under Cygwin, but found that the variability in timing on that platform was so large as to render the test results meaningless.  As an example, here are four successive runs on that platform:

All functions checked for accuracy; checking timing...
    original    1902.0000000000 0.000000% faster than original
       naive    2124.0000000000 11.671924% slower than original
      Edward    2182.0000000000 14.721346% slower than original
    dbasnett    2091.0000000000 9.936909% slower than original
  Jan Korous    1980.0000000000 4.100946% slower than original
     scottbb    2144.0000000000 12.723449% slower than original
         JS1    2276.0000000000 19.663512% slower than original
  JS1 Lookup    2075.0000000000 9.095689% slower than original
All functions checked for accuracy; checking timing...
    original    2174.0000000000 0.000000% faster than original
       naive    2121.0000000000 2.437902% faster than original
      Edward    2088.0000000000 3.955842% faster than original
    dbasnett    2055.0000000000 5.473781% faster than original
  Jan Korous    2136.0000000000 1.747930% faster than original
     scottbb    2109.0000000000 2.989880% faster than original
         JS1    1855.0000000000 14.673413% faster than original
  JS1 Lookup    2149.0000000000 1.149954% faster than original
All functions checked for accuracy; checking timing...
    original    2073.0000000000 0.000000% faster than original
       naive    2154.0000000000 3.907381% slower than original
      Edward    1960.0000000000 5.451037% faster than original
    dbasnett    2288.0000000000 10.371442% slower than original
  Jan Korous    2032.0000000000 1.977810% faster than original
     scottbb    2091.0000000000 0.868307% slower than original
         JS1    2245.0000000000 8.297154% slower than original
  JS1 Lookup    2138.0000000000 3.135552% slower than original
All functions checked for accuracy; checking timing...
    original    2158.0000000000 0.000000% faster than original
       naive    2277.0000000000 5.514365% slower than original
      Edward    1840.0000000000 14.735867% faster than original
    dbasnett    2081.0000000000 3.568119% faster than original
  Jan Korous    2135.0000000000 1.065802% faster than original
     scottbb    2228.0000000000 3.243744% slower than original
         JS1    1969.0000000000 8.758109% faster than original
  JS1 Lookup    2010.0000000000 6.858202% faster than original


Laravel controller method to update user

So the above answer from Sᴀᴍ Onᴇᴌᴀ already covers quite a few points.
I've worked with Laravel for the past 4 years, so I'll go a bit further in depth on how I'd approach this, including a few suggestions.
First you'll want to use RequestValidation as Sᴀᴍ Onᴇᴌᴀ indicated, this will help guard against invalid input. And make use of Laravel's own input validation logic (sends the user back to the form with error messages in your message bag)
Secondly you have a lot of logic on the User model related to an avatar. I'd suggest creating an Image / Avatar Model that handles the image part of it all, in the end you'd end up with an image_id/avatar_id on the User model as a relation. That way you can have all the logic that handles a file on your filesystem and the associated data. It also enables you (if you go for image) to reuse the logic in a different context.
And declutters your user model
Thirdly, as Sᴀᴍ Onᴇᴌᴀ indicated you do a lot of ->save() calls.
Each call is a database round trip. And a costly action.
You can set each property seperately if you want and at the end of your logic you can call ->isDirty() on your user model, if it returns true, data has changed from the database returned data, indicating an update should be done. At this point you could save.
I however would suggest building an update array of the data to update, and calling the ->update() method with an array containing the key => value pairs that you wish to update. You can check this data array for emptyness to avoid unneccessary updates.
I would always suggest having some form of validation when changing passwords, to make sure an unauthorized user can't simply change the password. (a simple enter current password would do here)
I'll refactor the code you had and try to leave it inside the existing method.
public function update(Request $request)
{
    $user = Auth::user();
    // Read up on the Laravel Request Validation for available rules and usages
    $request->validate([
        'username'         => 'sometimes',
        'email'            => 'email|sometimes',
        'wallet_address'   => 'sometimes',
        'about'            => 'sometimes',
        'password'         => 'sometimes',
        'current_password' => 'required_with:password',
        'website'          => 'sometimes',
        'location'         => 'sometimes',
    ]);
    // $request->input() will return null if the given key is not found
    // I use an array_filter here to filter out empty results, if you expect falsey values (like null to unset something, or 0/false/'') you should use a different approach
    $update_data = array_filter([
        'username'       => $request->input('username'),
        'email'          => $request->input('email'),
        'wallet_address' => $request->input('wallet_address'),
        'about'          => $request->input('about'),
        'password'       => $request->input('password'),
        'website'        => $request->input('website'),
        'location'       => $request->input('location'),
    ]);
    if ($request->input('password')) {
        // enter logic here to validate existing password
        // if this logic fails, you should just return instead of saving anything
    }
    if (!empty($update_data)) {
        // You can store the result to see if you successfully updated
        $update_status = $user->update($update_data);
    }
    // Check if an file has been given
    if ($request->hasFile('avatar')) {
        // I'd suggest moving the image handling to a seperate method at least.
        $this->handleImage($user, $request->file('avatar'));
    }
    if (!$update_status) {
        return redirect()->route('edit_profile')->with('failure','Your account could not be updated.')
    }
    return redirect()->route('edit_profile')->with('success','Your account has been updated successfully.')
}

private function handleImage(User $user, UploadedFile $image)
{
    // if the image is a seperate relation, you could delete the existing relation
    // on the delete for the relation you could hook in a cleanup for the file.
    if ($user->avatar_name) {
        // You can add a "disk" to your filesystem configuration where you can set the default path.
        Storage::delete('/public/profile-pics/' . $user->avatar_name);
    }
    $image_new_name = date('dmy_H_s_i') . '_' . $user->id . '_' . $image->getClientOriginalName();
    $image->storeAs('profile-pics', $image_new_name, 'public');
    // if you use a properly configured Storage::disk you can only store the name, and the disk will resolve where to look for the file
    // Storing paths in the database binds your database to your filesystem structure making future changes more difficult to handle (think moving images to a seperate location, or a different server)
    // Another point, if you expect many users, consider using a subdivision in your path (something like the first few digits of the user id or anything that'll let you break up the files into subdirectories), most systems do weird things once you deal with 200k+ files in a single directory.
    $user->avatar_url = 'storage/profile-pics/' . $image_new_name;
    $user->avatar_name = $image_new_name;
    $user->save();
}


How to check list of environment variable in python?

You can group the messages and corresponding exit codes into a dict, and then iterate over that one:

def check_mandatory_envs():
    exit_codes = {
        "ENVCONTEST_1" :125,
        "ENVCONTEST_2" :126,
        "ENVVIRTEST_3" :127,
        "ENVVIRTEST_4" :128,
        "ENVPATHTEST_5":129,
        "ENVPATHTEST_6":130,
    }
    for variable, code in exit_codes.items():
        if variable not in os.environ:
            exit(code)

If the order of the iteration is important and you are on python before 3.6, you canuse a collections.OrderedDict or a list of tuples, without the .items() call


Pick a random file from a directory tree

If we rewrite your code to take advantage of Pathlib, we get the following:

import os
import sys
import random
from pathlib import Path

def get_random_files2(ext, top=os.getcwd()):
    file_list = list(Path(top).glob(f"**/*.{ext}"))
    if not len(file_list):
        return f"No files matched that extension: {ext}"
    rand = random.randint(0, len(file_list) - 1)
    return file_list[rand]

if __name__ == '__main__':
    extension = sys.argv[1] if len(sys.argv) > 1 else "*"
    print(get_random_files2(extension))

I appended a "2" to the function name in my example just to make it easier to copy/paste if you want.
The benefits of using Pathlib is that we can glob directly from the path object, and it will recursively get a list of files throughout the tree.

I noticed your code only works with os.getcwd() so there was no point in injecting two parameters into get_random_files. However, I left the function signature the same in case you want to enhance it later.

The looping seemed out of place when (a) you want to check for files (b) you want a random file. So, the code gets the files or returns "No files found", and if it finds some files, picks one of them from the list of files.
Please take some time to understand the approach I used. The biggest take-aways you can have are:

Infinite loops are bad (the while True:) unless you really, really need them (because you don't want to write event-based code or you just want to block),
Looping is always slow,
Modifying variables inside loops is bad (your construction/approach is faulty when you modify variable content when those variables control or affect the functionality or processing of what your code is actually doing),
if you're going to use random values, it's best to understand why you're doing that. If it's to pick from a list, ensure you populate the list first, don't rely on the random value to dictate your actions.
Look at keeping functions to 4 or so lines - because this will ensure you adhere to the Single Responsibility principal, which makes your code better in the long run (less support and maintenance).

Otherwise, a good attempt at automation. Keep it up and post more of your code when you have it.


Best way to replace a beginning and end character in Perl using Regular Expression?

Here is a single regex that removes , (comma) at the beginig or at the end of a string:

$str =~ s/^,+|,+$//g;

and here is a benchmark that compares this regex with a double one:

use Benchmark qw(:all);

my $str = q/,a,b,c,d,/;
my $count = -3;
cmpthese($count, {
        'two regex' => sub {
            $str =~ s/^,+//;
            $str =~ s/,+$//;
        },
        'one regex' => sub {
            $str =~ s/^,+|,+$//g;
        },
    });

The result:

               Rate one regex two regex
one regex  597559/s        --      -58%
two regex 1410348/s      136%        --

We can see that two regex are really faster than one regex that combines the two.


How do I generate a list of n unique random numbers in Ruby?

The sample method does what you want:

(1..10).to_a.sample(7) #=> [2, 9, 1, 6, 8, 10, 4]


Build SQL query criteria from form input

If it works, it's good but it's not very easy to understand. It is a Loop-switch_sequence and can be seen as an anti-patern. See:
https://en.wikipedia.org/wiki/Loop-switch_sequence

If this is working, don't touch it, but if you have to simplify it, you can get rid of the loop and the switch and have just blocs like this.

if(!empty($inputs['inv_num']){
    if(!empty($constraints)){
       $constraints .= ' AND ';
    }
    $constraints .= "$invTable.$param='$val'";
}
//....... other variables name, customer_id, pcs ...
$constraints = $sql_conj . ' ' . $constraints;


Merging array entries referenced by indexes in another array

If you index the answers to the index of the questions you can do this:

let r_arr = {
            0: true,
            3: false
};
let q_arr = [
            { q: 'some question one', o: [true, false] },
            { q: 'some question two', o: [true, false] },
            { q: 'some question three', o: [true, false] },
            { q: 'some question four', o: [true, false] }
        ];

var a = q_arr.map((q, i) => ({'q': q.q, 'o': i in r_arr ? r_arr[i]  : 'na'}) )
console.log(a)

You'll need to account for the fact that js arrays are zero indexed.


Just some general syntax notes I have after a quick scan of your code...

Python isn't C, not all evaluations are about equality!

The line while playing == True:, immediately struck me as odd.  Python evaluates things in if-statements/while-loops/etc.  This is different to a language such as C, which always uses equality (or inequality) for such things.

Specifically, if we take the variable playing to just be (something that is either True or False), and then substitute that into the expression.. We get:

(something that is either True or False) == True

This will *ALWAYS* evaluate to the same value as just using playing - since it's already either True or False.

Input checking, and why (when dealing with users) even sane people validate

The following excerpt:

#gets the number of questions the player wants to answer
num = int(input("\nHow many questions would you like: "))

As you've discovered, input is returning you a string, and the way to cast this to an int is with (the appropriately named) int.  However, if a user enters something like "Tersosaur" instead of a number, you will get a ValueError (raised by int).

The way to deal with this is to use a try/except, like this:

#gets the number of questions the player wants to answer
num = input("\nHow many questions would you like: ")
try:
    num = int(num)
except ValueError:
    print "Error '%s' is not a valid number".format(num)

Just a recommendation,  Make your code more Pythonic

I noticed a lot (all) of your initialization and start-up code (things such as setting score = 0, etc) happens in the program itself, and not in a main block.

You don't technically need this, but it is best practise to use a block like this:

if __name__ == "__main__":
    ...initialization code goes here

Functional Decomposition is your frenemy friend

Functional Decomposition is important, even in small program like this.  It improves the readability, understandability and (perhaps most importantly here) maintainability of your code.

At the moment, all of your logic is wrapped up inside this mainloop style sequence of commands.  The way you get input from the user, the way you keep score, how you check if their answers are correct, etc.  All of these ways and hows are behaviours, which in Python can* become methods.

* Note: Functional Decomposition is not ALWAYS your friend (hence frenemy).  If you go to far you'll end up with just as big a mess of spaghetti code as you started with.  For example, don't extract each and every line from your loop into a separate function and think it's functionally decomposed.  Functional decomposition is about the identification of behaviours.


There's an XML syntax for DOT called DotML at http://martin-loetzsch.de/DOTML/. Generating a generic DotML tree from XML is very straightforward using XSLT. Most of it is just:

<xsl:template match="*">
  <node id="{generate-id()}" label="{name()}"/>
  <edge from="{generate-id(..)}" to="{generate-id()}"/>
  <xsl:apply-templates/>
</xsl:template>

<xsl:template match="text()">
  <node id="{generate-id()}" label="{.}"/>
  <edge from="{generate-id(..)}" to="{generate-id()}"/>
</xsl:template>


Some smaller adjustments

By just quickly glancing on the code I would suggest the following actions:

Move the opening of the output file to where you open the inSortedKey file, and let it be open until finished processing the files. Opening and closing files is rather expensive.
Instead of splitting the line multiple times, do it once in the start of searchWindow() and use the different elements when you need them. Possibly skip the intermediate variables all together
Simplify the logic related to the isHere if statements, as you do the same in both cases only that in the second case click will always be 0. Thusly eliminate the duplication of code here
Consider using another level of with open(...) and for another_line instead of the next((line for line ... )), and do early return in that loop to skip lines as soon as possible

A little style advise to close off: Remember that the Python style guide indicates to use snake_case for both variables and function names. :-)

Multiple reads of dataset

For each line in inSortedKey file you read all lines from each of the files in the dataset. That is a lot of extra reading of files!!! Lets say that you have \$n\$ lines in inSortedKeys and \$m\$ files in the dataset, where each file has approximately \$p\$ lines. Then your code does \$O(n\cdot m\cdot p)\$ operations within searchWindows.

With your numbers being: \$n=62.000.000\$, \$m=10\$, and \$p=16.000.000\$.

If you had been able to preread the inSortedKey and extract the keys into a memory table, and then do a single read on each of the other dataset files matching each line against this precomputed key list this number would drop to \$O(n + m \cdot p)\$. That would really speed up your operations, but it does depend on the size of inSortedKey.

Even if you are not able to load all of inSortedKey you would be much better of reading as large a chunk as you are comfortable with into memory, before doing a full read on the dataset. Say you read 100000 lines of inSortedKeys into memory, you would look at \$O(\frac{n}{100000} \cdot m \cdot p)\$ operations doing a slightly heavier operation each time. It would still be a massive improvement.

Consider switching to databases

If you're actually talking about cross-referencing 62 million keys against 160 millions lines in the dataset files, you should seriously consider using databases.

Either permanently, or temporarily. You would most likely get a performance gain if you imported the entire set into a database, did your operations, and wrote the output rather than doing this with plain file operations. Python has excellent support for sqlite or pandas, in addition to multiple other modules and databases, which might help you.

See 200_success' answer for a somewhat related discussion on massive file sizes.


Welcome to Code Review and thanks for sharing your code!

General issues

Naming

Finding good names is the hardest part in programming. So always take your time to think carefully of your identifier names.

Naming Conventions

It looks like you already know the
Java Naming Conventions.

Avoid abbreviations

In your code you use some abbreviations such as  maxSP and baseMeleeDmg.
Although this abbreviation makes sense to you (now) anyone reading your code being not familiar with the problem (like me) has a hard time finding out what this means.

If you do this to save typing work: remember that you way more often read your code than actually typing something. Also for Java you have good IDE support with code completion so that you most likely type a long identifier only once and later on select it from the IDEs code completion proposals.

  The other identifiers make perfect sense to me. [because of context] – AJD

Context looks like your friend but in fact it is your enemy.

There are two reasons:

Context depends on knowlegde and experience. But different persons have different knowlegde and experience, so for one it might be easy to remember the context and for another it might be hard. Also your own knowledge and experiences change over time so you might find it hard to remember the context of any given code snipped you wrote when you come back to it in 3 in years or even 3 moth.

The point is that in any case you need (more or less) time to bring the context back to you brain.
But the only thing that you usually don't have when you do programming as a business ist time. So anything that makes you faster in understanding the code is a money worth benefit. And not needing to remember any context is such a time saver.
You may argue that we have a very simple promblem with an easy and common context. Thats true.
But:

Real life projects usually have higher complexity and less eays to remember contexts. The point here is:
At which point is your context so complext that you switch from "acronym naming" to "verbose nameing"?

Again this point changes with you knowledge and your experience wich may lead to code that others have a hard time to understand.

The much better way to deal with it is to always white your code in a way that the dumbest person you know may be able to understand it. And this includes not to use akronyms in your identifiers that might need context to understand.
This is a training project. When you train a physical skill like Highjumping you start with a very low bar that you can easily pass even not using the flop technique just to have a safe environment.

Same is here: The problem may be simple enough to be understood having the acronymed identifiers, but for the sake of training you should avoid acronyms.

Avoid misleading naming

Both of your method names are misleading: They claim to create and build something but in reality neither one is creating or building anything.

One method is doing user interaction and the other is configuring the object.

The names of the methods should reflect that.

Add units to identifiers for physical values

Physical values do mean nothing without a unit. This is a special case of the context problem mentioned above.  The most famous example is the fail of the two space missions Mars Climate Orbiter and Mars Polar Lander. The Flight control software was build by NASA and worked with metric measurement (i.e. meters, meters per second or newton seconds) while the engine (and their driver software) has been build by Lockheed Martin which use imperial units (i.e. feet, feet per minute or Pound-force second).

The point is: not having units of physical values in your identifiers forces you to think if there is a problem or not:

double acceleration = flightManagement.calculateAcceleration();
engine.accelerate(acceleration);

But usually you don't quesion it unless you have a reason...

Having the units in the identifier names the problem becomes obvious:

double meterPerSquareSeconds =
          flightManagement.calculateAccelerationInMeterPerSquareSecond();
engine.accelerateByFeedPerSquareMinute(
          meterPerSquareSeconds); // oops

And we have the same argument again: your particular code is so easy and so small that we don't need the overhead.
But then: How do you decide at which point the overhead is needed? And again it depends on knowledge and experience which still are different among people...

Flawed implementation

Both methods using the same member variables.
E.g. you variable   maxSP: in createCustomClass() you assign it the result of kb.nextInt(). Then you pass this value as parameter to buildCustomClass() where you again assign the parameter value  again  to the same  member variable.

Beside this being useless it may lead to confusing bugs later.

keep same level of abstraction

Methods should either do "primitive" opeerations or call other methods, not both at the same time.

At the end of your method createCustomClass() you call the other method (buildCustomClass()). The better way to do this is to extract the code before the call to buildCustomClass() in a separate (private) method:

public void createCustomClass() {
    aquireDataFromUser();
    buildCustomClass(
          maxHP,
          maxMP,
          maxSP,
          baseMeleeDmg,
          baseSpellDmg,
          baseAC,
          baseSpeed);
}

private void aquireDataFromUser() {
    kb = new Scanner(System.in);

    System.out.println("Enter the information for your class: ");
    System.out.println("Enter HP: ");
    maxHP = kb.nextInt();
    System.out.println("Enter MP: ");
    maxMP = kb.nextInt();
    System.out.println("Enter SP: ");
    maxSP = kb.nextInt();
    System.out.println("Enter Base Melee Damage: ");
    baseMeleeDmg = kb.nextInt();
    System.out.println("Enter Base Spell Damage: ");
    baseSpellDmg = kb.nextInt();
    System.out.println("Enter AC: ");
    baseAC = kb.nextInt();
    System.out.println("Enter Speed: ");
    baseSpeed = kb.nextInt();
}

Beside making createCustomClass() shorter it makes the useless reassingment of the member variables obvious.


Sieving in the sense of the old Greek is about eliminating composites by crossing out all multiples of primes (striding additively), and the original SoE does not divide, ever. Trial division is something completely different, and for numbers approaching 2^64 it takes several million times as long as a segmented Sieve of Eratosthenes, even if you trial-divide only by primes.

The memory footprint can be reduced by sieving only the odd numbers, since the only even prime can be pulled out of thin air if need be.

Sieves are usually represented as packed bitmaps rather than one number per array cell, because otherwise memory consumption would be huge and things would be exceedingly slow. More compact forms of representations exist, which extend the 'odds-only' idea to dropping multiples of more small primes. They are called 'wheels', and one of the regulars here has written a brilliant explanation of them.

The famous mod 30 wheel drops 2, 3, and 5, and thus effectively stuffs 30 integers into a single byte. Packed odds-only bitmaps are much less complicated than such wheels though and good enough for many practical purposes.

The biggest speedup - almost an order of magnitude - comes from sieving in small blocks that fit into the L1 cache of the CPU, 32 KBytes or thereabouts. However, this makes division creep back into the code, because it is necessary for computing the start offset for each prime when processing a segment.

The division for recomputing the striding position for a prime for every new segment can be eliminated by remembering the last offsets for each prime from segment to segment (or rather the next offsets). That results in a further speedup.

The last optimisation is called 'presieving', which in practical terms means blasting the bit pattern for the composites of a bunch of small primes all over a bitmap before commencing normal operation at the first prime not in that bunch.

There are two variations: the general one blasts a segment immediately before commencing work on it, thus also warming the caches. The special one blasts the whole sieve in one go, and it is slightly faster overall given certain conditions. Perfect for initialising an auxiliary small factors sieve (up to 2^32) for a sieve that operates up to 2^64.

My pastebin has several standalone .cpp programs demonstrating these optimisations and their effects, so that you can gauge what each optimisation can buy you in terms of speed. Presieving might be slightly more difficult to do in python but all the other optimisations apply directly.

As a rough guide, sieving the full 32-bit range using the plain odds-only Eratosthenes takes 20 to 30 seconds on modern CPUs (single-threaded). If you apply all the optimisations then you can cut that to 2 seconds. The segmented sieve logic is also a precondition for parallel sieving.

EDIT: if you are interested in primes in general and large lists - or fast bulk generation - in particular then have a peek at my topic
Checksumming large swathes of prime numbers? (for verification). It mentions quite a few sources, up to 2^64 - 10 * 2^32 in fast bulk, from then on at a rate of about 1 million primes per minute (by having gp/PARI dump forprime()). primos.mat.br has lists up to 10^12, by the way.

EDIT: some resources on how to do Sieve of Eratosthenes in python and who to apply some of the optimisations:

Gareth Rees covers the basics in his excellent post under Sieve of Eratosthenes - Python
the python SoE examples on RosettaCode show many variants and some optimisations in action
the numpy/scipy docs have the lowdown on fancy indexing and dtype (to force arrays to specific data types like unsigned 32-bit integers a.k.a. uint32)
LiterateProgramming explains the basics of SoE in python at a gentler pace

The keys to speedy sieving in languages like python are two:

efficient, compact representation of the sieve (sets, bool arrays, uint32/uintp arrays)
pushing processing into the engine/runtime without incurring gratuitous inefficiencies (e.g. instead of division filter where(n % p != 0), use additive striding like is_composite[p*p::p] = True for a bool array)

Things like presieving pretty much require using uintp arrays to be really effective but sets and boolean arrays are more interesting for exploring the logic of the Sieve vs. the capabilities of python, numpy and scipy.

Note: your use of the division filter turns your code into 100% bona fide trial division, and it would make the old Greek turn in his grave... ;-)


Each of these:

string account = string.Empty;

if (queryWhere.ContainsKey("account")
   && queryWhere["account"] != null
   && !string.IsNullOrEmpty(queryWhere["account"].ToString()))
{
    account = queryWhere["account"].ToString();
}

Should be reduced to:

string account;
if(!queryWhere.TryGetValue("account", out account))
{
   account = string.Empty; // if you need the string to be empty // default is null
}

Dictionary.TryGetValue

But that logic can then be moved to a method:

private string GetValue(string key)
{
   string returnValue;
   if(!queryWhere.TryGetValue(key, out returnValue))
   {
      returnValue= string.Empty;
   }
   return returnValue;
}

string account = GetValue("account");
string customer = GetValue("customer");

I really don't see the point of your original code, BTW. For instance, the .ToString() is completely superfluous, since you're working with a Dictionary<string,string>. It is always going to return a string.

But why do you even check for string.IsNullOrEmpty()? You already know it isn't null from the previous line's check -- queryWhere["account"] != null -- so at worst it is empty which is your default value anyway!


Risk board game - best practices, private vs public variables and methods

If you stop worrying about "privates" everything becomes much cleaner.

By using closure with my "country" object am I wasting memory, and hence should be using prototypical inheritance?

The closure is a small performance hit. For a board game it won't matter. But if you decide not to use local variables in closures (AKA "privates") then you can get rid of the closure and use a constructor function / prototype.
One benefit of doing that is that it will give tools like eclipse, webkit inspector, etc. a clue as to how things are represented in your code so they can annotate stuff for you, provide pop-up help, autocompletion, things like that.

My jsonify code seems redundant, I assume there's a more correct way to do this?
Mixing private instance variables like "countries" and public instance variables looks ugly, is that how it's supposed to be done?

Getting rid of "privates" and making these all properties would be the easiest way to fix both of these issues. In that case the get_* functions could also go.
A few other points:

Usually javascript identifiers are written as camelCase, not underscore_separated.

The quotes around the property names in the object literal are not needed for character sequences that would otherwise be valid identifiers.

to is a confusing variable name, try to find something more descriptive. I changed it to country in the example below.

I would probably write it something like this:
function Country (name, connectedTo){
    this.name = name;
    this.connectedTo = connectedTo;
    // other constructor stuff...
}

Country.prototype = {

    // default properties

    numTroops: 0,

    player: null,

    // functions

    isOwnedBy: function(inPlayer){
        return inPlayer == this.player;
    },
    isTouching: function(country){
        // best to avoid typeof if possible
        if (country.name){
            country = country.name;
        }
        // this performs somewhat better than evaluating foo.length every iteration
        for (var i = 0, len = this.connectedTo.length; i < len; i++){
            if (this.connectedTo[i] === country){
                return true;
            }
        }
        return false;
    },
    setState: function(inPlayer, inNumTroops){
        this.player = inPlayer;
        this.numTroops = inNumTroops;
    },
    toString: function(){
        return '[country '+this.name+']';
    },
    getAscii: function(){
        return this.name + ' owned by ' + this.player + ' with '+ this.numTroops +
            '\n        connected to ' + this.connectedTo;
    }
};

If setState is only called once per object, you might consider making that code part of the constructor and getting rid of setState.


Reading analog moisture signal from Arduino over USB to RPi3 and publishing via paho MQTT

Here're some changes that I'd personally do:
PEP8 Guidlines

Be consistent when writing code! You used both camelCase and snake_case conventions when you declared your functions. Chose one of them and keep it that way. PEP8 recommends the second one.
Talking about consistency, you should also have a space after each #. More, when you want to write what your function is doing, you should use the following:

def read_sensor(choice):
    """
    Read analog moisture data from moisture probe via Arduino UNO plugged into
    the RPi via USB
    """
    # ...

After each comma (or any other operator) you should have a space
This: if moisture <= alert_threshold and moisture > alert_low: can be simplified using a chained expression:

if alert_low < moisture <= alert_threshold:
    ....

Use string format() instead of using the old + operator.
Constants should be UPPER_CASED (but not inside functions). I just took them out.

Personal anecdote:

It feels a bit weird to have the main() function at the beginning and everything within that function after. I'd just move it to the end as it will look more naturally.
You have several magic numbers which should be declared at the beginning. Doing this, you'll be able to modify their value faster than going through all the code and trying to remember what does what.
Don't store variables if you use them only once. It will make you're code harder to follow and it will speed it down (don't think of it as a big speed down but even so...)

The above will lead our code to the following:
import serial
import time
import RPi.GPIO as gpio
import paho.mqtt.client as mqtt

ALERT_LOW = 10
ALERT_HIGH = 90
ALERT_TRESHOLD = 30
RELAY_TIME = 14
TIME_TO_SLEEP = 60 * 1
MIN = 330
MAX = 970
RATE = 9600

def read_sensor(choice):
    """
    Read analog moisture data from moisture probe via Arduino UNO plugged into
    the RPi via USB
    """
    ser = serial.Serial('/dev/ttyACM0', RATE)

    # Line below handles the problem of the Arduino returning more than 1 reading
    raw_data = int(str.split(ser.readline())[0])
    if choice == "raw":
        moisture = raw_data

    # Stop readings going over or under 0 or 100%
    if raw_data < MIN:
        raw_data = MIN
    if raw_data > MAX:
        raw_data = MAX
    if choice is "raw_adjusted":
        moisture = raw_data
    if choice == "clean":
        # Get moisture as  % reading
        moisture = int(round(100 - (100 / float(MAX - MIN)) * (raw_data - MIN), 0))

    return moisture

def send_mqtt(output, topic):
    """Publish to MQTT for given topic"""
    host, user, password, port = "MQTTBroker.local", "user", "password", 1883

    mqttc = mqtt.Client()
    mqttc.username_pw_set(user, password)
    mqttc.connect(host, port)
    mqttc.publish(topic, output)

def alerts():
    """ Generates alerts that are humanly readable for certain moisture thresholds"""
    moisture, output = read_sensor("clean"), ''

    if moisture <= ALERT_LOW:
        output = "**Moisture is below 10%**"
    if ALERT_LOW < moisture <= ALERT_TRESHOLD:
        output = "***Moisture is between 10%` and 30%, watering***"
        relay(RELAY_TIME)
        time.sleep(TIME_TO_SLEEP)
    if moisture >= ALERT_HIGH:
        output = "***Moisture is above 90%***"
    send_mqtt(output, "Planty/alerts")

def debugger():
    """This is just to test each reading"""
    output = "Soil Moisture is: {}%\n" \
             "Raw_adjusted is: {}\n" \
             "Raw is: {}".format(read_sensor("clean"), read_sensor("raw_adjusted"), read_sensor("raw"))

    send_mqtt(output, "Planty/debug")

def relay(pin):
    """Turn on relay via PIN.HIGH"""
    try:
        gpio.setmode(gpio.BCM)
        gpio.setwarnings(False)
        gpio.setup(pin, gpio.OUT, initial=1)
        send_mqtt("Turning Relay1 ON", "Planty/relay")
        gpio.output(pin, 0)
        time.sleep(5)
        send_mqtt("Turning Relay1 OFF", "Planty/relay")
        gpio.output(pin, 1)
    finally:
        gpio.cleanup()

def main():
    try:
        while 1:
            send_mqtt(read_sensor("clean"), "Planty/moisture_readings")
            alerts()
    except KeyboardInterrupt:
        print("\n\nExiting via Keyboard Interrupt")

if __name__ == '__main__':
    main()

Since you're using the try/except in main only for KeyboardInterrupt I'd completely remove it as it seems redundant.
def main():
    while 1:
        send_mqtt(read_sensor("clean"), "Planty/moisture_readings")
        alerts()


I noticed that the face_recognition library itself is quite optimized to run on both multiple cpus and gpu as well. But I will try to give my two cents for your specific code here.

Typically, there are two steps when it comes to such applications.

There is the training step where you parse all the known_person images once, and build a data of face_encodings, and store them on the disk. Typically these encodings are much smaller than the image data itself in size and it is something you do once for all the known images. (if you add new known images, consider this as a recalibration step where you need to regenerate your encodings). This step is usually accepted to be time-consuming and not done very frequently.

Then there is the inference step, which is the compare_faces function. This is what you want to do several times. For this specific step you just need the new image and known_list(which is the trained data that you stored on disk ahead of time).

So to improve your library, I would suggest adding a 'train' function which allows a list of images. This list should then store the encodings in a format (compressed if you have several images).

Then the compare_faces function can be modified to take in the encodings data. You can split this into load step (in case you compressed the data) and then compare step. This allows application to load the data once, but infer several times.

In your code itself, the compare_faces can be improved already. It can be changed to:

def compare_faces(unknown_images, image_encodings_data):
  for image in unknown_images:
    unknown_data = create_face_comparison_encoding(image)
    # Note how compare_faces takes multiple encodings.
    results = face_recognition.compare_faces(image_encodings_data,unknown_data)
    if results.count(True) > 0:
      print("This person appears familiar!")
      print("Writing the faces to output folder!")
      recognize_faces(image)
    else:
      print("This doesn't appear to be familiar!")

Also, I find it strange that you do not return which face it actually matched to.

Please feel free to ignore me if this is no longer your concern :)


The task is about dynamic arrays, and using an array (instead of a
dictionary, as you did) simplifies things considerably.

If you start with an array of numOfSequences empty arrays:

var seqList = [[Int]](count: numOfSequences, repeatedValue: [])

then the 1 x y query simplifies to

let seqIndex = ((x ^ lastAns) % numOfSequences)
seqList[seqIndex].append(y)

without the need to check for an empty dictionary value.

Some more suggestions:

Reading a list of integers occurs at two places in the program, that
justifies a separate function:

func readIntegers() -> [Int] {
    return readLine()!.componentsSeparatedByString(" ").map { Int($0)! }
}

(Remark: Usually, forced unwrapping with ! should be avoided,
but here we have a programming challenge with well-defined input data.)

To increase the legibility of the code, I would assign the three
parameters of a query to variables type, x, y.
The code then becomes almost self-explaining.

The entire code then looks like this:

func readIntegers() -> [Int] {
    return readLine()!.componentsSeparatedByString(" ").map { Int($0)! }
}

// Read  N(number of sequences) and Q(number of queries):
let input = readIntegers()
let (numOfSequences, numOfQueries) = (input[0], input[1])

var lastAns = 0
// Array of `numOfSequences` empty arrays:
var seqList = [[Int]](count: numOfSequences, repeatedValue: [])

for _ in 0..<numOfQueries {
    let input = readIntegers()
    let (type, x, y) = (input[0], input[1], input[2])

    switch type {
    case 1:
        let seqIndex = ((x ^ lastAns) % numOfSequences)
        seqList[seqIndex].append(y)
    case 2:
        let seqIndex = ((x ^ lastAns) % numOfSequences)
        let index = y % seqList[seqIndex].count
        lastAns = seqList[seqIndex][index]
        print(lastAns)
    default:
        fatalError("Unexpected query")
    }
}


readAllInts

This is a wishy washy name. What does it mean, "all ints"? In the universe? In my little black book of ints? Entered by the user on standard input? Aha!

It might be good to split this up to multiple methods:

a method to read and return the number of ints from a scanner and handle errors: print a message in case of invalid input and try again, or simply throw an exception
a method to read specified number of ints from a scanner and return as an array
a method that calls the previous two and returns the array

That way each method will have a simpler, single responsibility.

A bigger problem with this method is that it doesn't belong in the InsertionSort class. The way you read an array of integers has nothing to do with the spring algorithm.

Type safety

By not using type tokens, you sacrificed type safety. For example a call less(4, "hello") would compile, which is undesirable.

You should never use bare types, always specify type parameters.
For example you can make less type safe like this:

private static <T extends Comparable<? super T>> boolean less(T v, T w) {
    return v.compareTo(w) < 0;
}

This effectively prevents less(4, "hello") at compile time.

Correcting sort similarly:

public static <T extends Comparable<? super T>> void sort(T[] a) {
    int length = a.length;
    for (int i = 0; i < length; i++) {
        for (int j = i; j > 0; j--) {
            if (less(a[j], a[j - 1])) { // Invariant broken, fix it.
                exch(a, j, j - 1);
            }
        }
    }
}

And that's it, the other methods are fine,
as they don't use bare types, and they are fine using Object[] for swapping and printing.

Naming

exch is not a great name. " exchange " would be better, but the most common name for this operation is "swap".

less is also not great, as some might interpret it in the sense of " minus ". I suggest "lessThan" instead, which makes the comparison logic perfectly clear.

The variable a is the worst offender of all. "swap" is not great either.


Im not familiar with XNA or have any experience working with matrixes, vectors and that kind of stuff. But i've made som basic refactorings to make it abit easier to read and maintain.

public static void GenerateCube(float size, out Vector3[] vertices, out Vector2[] texCoords, out Vector3[] normals)
{
        var vertexList   = new List<Vector3>();
        var texCoordList = new List<Vector2>();
        var normalList   = new List<Vector3>();

        const float ninety = (float)(Math.PI / 2);

        for (var i = 0; i <= 2; i++)
        {
            var normal = CreateVector(i);
            var matrix = CreateMatrix(i, ninety);

            var vectorsA = CreateVectorsA();
            var vectorsB = CreateVectorsB();

            vectorsA.ForEach(x => Vector3.Transform(x, matrix));
            vectorsB.ForEach(x => Vector3.Transform(x, matrix));

            for (var j = -1; j <= 1; j += 2)
            {
                var vectors = j == -1 ? vectorsA.ToArray() : vectorsB.ToArray();

                var someCalculations = CalculateSomething(j, normal);
                normalList.AddRange(someCalculations);

                AddTextureCoordinates(texCoordList);
                AddVertexes(j, vertexList, vectors);
            }
        }

        vertices  = vertexList.ToArray();
        texCoords = texCoordList.ToArray();
        normals   = normalList.ToArray();

        SetVectorSize(vertices, size);
    }

    private static void SetVectorSize(IList<Vector3> vertices, float size)
    {
        for (var i = 0; i < vertices.Count; i++)
        {
            vertices[i] *= size;
        }
    }

    private static Vector3 CreateVector(int i)
    {
        return i == 0 ? Vector3.UnitZ : (i == 1 ? Vector3.UnitY : Vector3.UnitX);
    }

    private static Matrix4 CreateMatrix(int i, float ninety)
    {
        return i == 0 ? Matrix4.Identity : (i == 1 ? Matrix4.CreateRotationX(ninety) : Matrix4.CreateRotationY(ninety));
    }

    private static void AddVertexes(int j, ICollection<Vector3> vertexList, IList<Vector3> vectorsArray)
    {
        //Triangle 1
        vertexList.Add(vectorsArray[0] * j);
        vertexList.Add(vectorsArray[1] * j);
        vertexList.Add(vectorsArray[2] * j);

        //Triangle 2
        vertexList.Add(vectorsArray[0] * j);
        vertexList.Add(vectorsArray[2] * j);
        vertexList.Add(vectorsArray[3] * j);
    }

    private static void AddTextureCoordinates(ICollection<Vector2> texCoordList)
    {
        //Triangle 1
        texCoordList.Add(new Vector2(0, 0));
        texCoordList.Add(new Vector2(0, 1));
        texCoordList.Add(new Vector2(1, 1));

        //Triangle 2
        texCoordList.Add(new Vector2(0, 0));
        texCoordList.Add(new Vector2(1, 1));
        texCoordList.Add(new Vector2(1, 0));
    }

    private static IEnumerable<Vector3> CalculateSomething(int j, Vector3 normal)
    {
        var someCalculation = normal * j;
        return Enumerable.Repeat(someCalculation, 6);
    }

    private static List<Vector3> CreateVectorsB()
    {
        return new List<Vector3>
                   {
                       new Vector3(-1, -1, 1),
                       new Vector3(1, -1, 1),
                       new Vector3(1, 1, 1),
                       new Vector3(-1, 1, 1)
                   };
    }

    private static List<Vector3> CreateVectorsA()
    {
        return new List<Vector3>
                   {
                       new Vector3(-1, -1, 1),
                       new Vector3(-1, 1, 1),
                       new Vector3(1, 1, 1),
                       new Vector3(1, -1, 1)
                   };
    }


Convert and print Celsius to Fahrenheit from certain range of values

The program works correctly, but there are a number of things you might be able to improve here.  Here are some observations in no particular order.

Reduce the number of variables

There are a lot of variables here, which makes it a bit difficult to read and understand the program.  Some of the suggestions below will show some ways to reduce that number and add more clarity to the program.

Use const where practical

A number of places in the code should have the const keyword added.  For example instead of this:

float count[4] = {10,1,10,100};
float bp[4] = {-50,-10, 10, 100};

write this:

const float count[4] = {10,1,10,100};
const float bp[4] = {-50,-10, 10, 100};

This indicates, both to the compiler and to human readers of the code, that these values will not be altered.

Use more meaningful variable names

The variable names step, min and max are useful and descriptive of what those variables actually do.  However, count and bp are not very good names.  I'd suggest calling them something more closely related to their function as with the following suggestion.

Consider using a struct

The count and bp values are closely related, but this is not clearly indicated in the code.  It would be easier to understand, I think, if a structure were used instead:

struct {
    float startTemp;
    float stepSize;
} const limits[] = {
    { -50,  10 },
    { -10,   1 },
    {  10,  10 },
    { 100, 100 },
    {1000,   0 },
};

Note that the minimum and maximum values are built into the table so there isn't any need for additional variables for those uses.

Declare variables in the smallest practical scope

By declaring variables in the smallest practical scope, you reduce the chance for name collisions and make it clear to the reader of your code where variables are and are not needed.  With any C compiler conforming to the 1999 specification (which should be all of them at this point!) one could rewrite the outer for loop like this:

for (float cels = min; cels <= max; cels += step) {
    float fahr = cels * 1.8 + 32;
    /* etc.*/
}

Initialize variables when they are declared

If you follow the advice above about reducing the scope of variables, you can also easily initialize variables when they are declared.  This helps to make sure that variables aren't used with uninitialized values.

Declare each variable on its own line

Declaring variables like this:

float fahr, cels, step, min, max;

is not good current practice.  Instead, declare one variable per line and initialize it.

Use the sizeof operator

These two lines are quite peculiar:

bp_size = *(&bp + 1) - bp;
st_size = *(&count + 1) - count;

A better way to find the size of such arrays would be this:

size_t bp_size = sizeof(bp);

Don't test floating point numbers for equality

One of the lines in the current code is this:

if (cels == bp[c]){

It will probably work just fine in this context, but generally, one should avoid testing floating point numbers for equality.  It's usually a better bet to test instead for < or <=.  See this question for more details.

Put if and for code on separate line(s)

The code contains this line:

if (c2 == st_size){ c2 = 0;}

It's not technically wrong and the compiler will accept it, but it's too easy for a human reader to overlook the contents of the if when it's hiding at the end of the same line.  Prefer instead to write it like this:

if (c2 == st_size) {
    c2 = 0;
}

Think carefully about loops

In this code, the outer loop seems clear enough, but the inner loop is not.  The point of it is to determine what step size to use but the code to do this is more complex than it needs to be:

for(c = 0; c < bp_size; c++){
    if (cels == bp[c]){
        if (c2 == st_size){ c2 = 0;}
        step = count[c2];
        c2++;
    }
}

First, we could just keep track of the step size as we go instead of searching for it each time.  Second, once the step size is found, there's no reason to continue through the rest of the loop.

Here's an alternative implementation that uses all of these ideas:

int main(){
    struct {
        float startTemp;
        float stepSize;
    } const limits[] = {
        { -50,  10 },
        { -10,   1 },
        {  10,  10 },
        { 100, 100 },
        {1100,   0 },
    };

    for (int i=0; limits[i].stepSize != 0; ++i) {
        for (float cels = limits[i].startTemp;
            cels < limits[i+1].startTemp;
            cels += limits[i].stepSize)
        {
            float fahr = cels * 1.8 + 32;
            printf("%4.0f C\t= %6.1f F\n", cels, fahr);
        }

    }
}

Note that the last item in the list is 1100 degrees rather than 1000.  This is done because we want to include the value of 1000 in the list.


Calculating when trains will meet, iteratively and using OOP

Note that pch.h header is not standard, and probably should not be part of this review.
Don't do using namespace std;.
Use initializer lists. That is, the constructor should be written as Train::Train(int loc, int vel) : location(loc), velocity(vel) { }. For POD types this might not matter, but it is idiomatic.
Because int getLocation() does not modify the state of the object, it should be made const, i.e., int getLocation() const.
In your main program, don't strive to declare all variables as soon as possible as this is C++ and not C. Instead, introduce variables as late as possible and as close to their site of usage as possible. As a remark, you should also initialize e.g., time when you define it to improve readability.
Yes, you could use a while-loop, but the for-loop has the advantage that you have to make an explicit decision as to what happens at the end of the loop in order for the program to compile. Because you need the time variable outside of the loop, you could do say for(; trainA.getLocation() < trainB.getLocation(); ++time) { ... }. A while-loop is also perfectly fine, and this boils down mostly to personal preference.
Perhaps you should enforce the train speeds to be positive. Otherwise you can get stuck in an infinite loop as the trains will never meet.
Your comments are too verbose. Good code is self-documenting via variables names and logical structure. Prefer comments that answer "why" to "what". For example, a comment like "input", "initialize each train", and "run the sim" are unnecessary, and only hurt readability.


First of all separate code into header (.hpp) and implementation file (.cpp)

Header : (my_header.hpp)

#ifndef MY_HEADER
#define MY_HEADER
class KeyInputParser
{
public:
    KeyInputParser  () // there is no need for funcName (void) in C++
    ~KeyInputParser () // that is neccessary only for C
    char* DIV;
    char* GKY;
    char* UID;
    void SetKeyString (const char* str);
    // Progress bewteen div, gky and uid
    char* Progress (char* previous);
    // Get data between <>
    void ExtractField (char* small, char* big);
};
#endif

Implementation : (my_code.cpp)

#include <cstring>
#include "my_header.hpp"

KeyInputParser::KeyInputParser ()
{
    //DIV will always be 16bytes... but it is either not required
    DIV = new char[32];
    //uid will ALWAYS be 7 bytes and is required
    UID = new char[14];
}

KeyInputParser::~KeyInputParser ()
{
    delete[] DIV;
    delete[] GKY;
    delete[] UID;
}

char* KeyInputParser::Progress (char* previous)
{
    char* arr = (char*) memchr (previous, '>', strlen (previous));
    arr++;
    return arr;
}

void KeyInputParser::ExtractField (char* small, char* big)
{
    for(int i = 0; i < strlen (small); i++)
    {
        if(small [i] == '<')
        {
            memcpy (big, small, i);
            big[i] = '\0';
            break;
        }
    }
}
void KeyInputParser::SetKeyString (const char* str)
{
    char* div = Progress (str);
    char* gky = Progress (div);
    char* uid = Progress (gky);

    ExtractField (div, DIV);
    // Code for GKY will be copied due to allocation
    // It might be possible to fix it to be used with ExtractField function
    for(int i = 0; i < strlen (gky); i++)
    {
        if(gky[i] == '<')
        {
            GKY = new char[i+1];
            memcpy(GKY, gky, i);
            GKY[i] = '\0';
            break;
        }
    }
    ExtractField (uid, UID);
}

This is just a rough refactoring of your code.
The thing number one that will need to be fixed is to drop all those C string functions and use std::string instead.

I did not test the code so I cannot guarantee that it will work but this is how it should look like. Hope I helped :)


Taking YouTube links out of a list of GitHub repo READMEs

Here are some concerns/suggestions:

you are reading the file twice - once to get the row count and when reading the links. And, you don't need to initialize the csv.reader to get the row count, simply use sum() over the lines in the file. You would probably need to use infile.seek(0) after getting the count and before initializing the csv reader
use _ for the throw-away variables (when counting the number of lines)
if len(ids) == 0: can be simplified as if not ids:
it looks like you don't need .findall() and should use .search() method since you are up to a single match
if there is a single repository link per line, you probably should have get_repository() method instead of get_repositories() and avoid the for repo in get_repositories(row[3]): loop - remember, "Flat is better than nested"
instead of handling the enumeration with row_counter manually, use enumerate()
instead of accessing the current row fields by index - e.g. row[1] or row[3], you can unpack the row in the for loop, something like (an example, I don't know your actual CSV input format):

for index, username, _, github_link in reader:

Or, you can use a csv.DictReader - accessing the fields by column names instead of indexes would improve readability - e.g. row["github_link"] instead of row[3]
you don't have to convert the args to a dictionary - return args and then access the arguments using a dot notation - e.g. args.filepath


Repetitive event handler for a toggling UI control

// Contact
var contactHeight = $('#contact-keebs').outerHeight();

function in_keebs() {
    $('#contact-keebs').fadeIn(200);
    $('body').css({
        height   : contactHeight,
        overflow : 'hidden'
    });
    $('#contact-info, #clients').addClass('animated fadeInUp');
}

function out_keebs() {
    $('#contact-info, #clients').removeClass('animated fadeInUp');
    $('body').css({
        height   : '',
        overflow : ''
    });
    $('#contact-keebs').fadeOut(200);
}

function show_project_button(e) {
    e.preventDefault();

    $('#contact-button').addClass('project-button').css('width', '71px').text('Projects');
    $('.mail-icon').attr('src', site.theme_path + '/img/icon-projects.png');
    $('#contact-button').shuffleLetters({callback:function(){
        if ($(window).scrollTop() != 0) {
            $('html, body').animate({
                scrollTop : 0
            },200, in_keebs);
        } else {
            in_keebs();
        }
    }
    });

    $('#mail-wrap').click(hide_project_button);
}

function hide_project_button(e) {
    e.preventDefault();

    $('#contact-button').removeClass('project-button').css('width', '96px').text('Get in touch');
    $('.mail-icon').attr('src', site.theme_path + '/img/icon-contact.png');
    $('#contact-button').shuffleLetters({callback:function(){
        if ($(window).scrollTop() != 0) {
            $('html, body').animate({
                scrollTop : 0
            },200, out_keebs);
        } else {
            out_keebs();
        }
    }
    });

    $('#mail-wrap').click(show_project_button);
}

$('#mail-wrap').click(show_project_button);

First, I'm not at all convinced that these are the right function names, but I needed names and these at least seem somewhat related.  Part of the problem is that I don't really understand what you're doing, as I don't know what the underlying HTML is.

The insight to get rid of the outer if/else is that you are simply alternating between them.  Instead of using a variable for tracking, we change the onclick behavior at the end of each variant.  We pick one and start with it.  This gives us the same alternating behavior as the if/else did.  We can also get rid of c.

We're still stuck with the inner if/else blocks, but we can remove the repeated code into their own functions.

Note that I haven't tested this, as I don't have enough context to run it.


The biggest issue I see is you have to manually register each and every command. Why bother? Why not just get a list of all commands that implement ICommand through Reflection? Speed shouldn't be a huge problem because I assume you do this on some sort of start-up.

var baseCommandInterface = typeof(ICommand);
var commands = AppDomain.CurrentDomain.GetAssemblies()
    .SelectMany(s => s.GetTypes())
    .Where(p => baseCommandInterface.IsAssignableFrom(p));

Now just enumerate commands and cast and operate on them. Life should be a lot easier this way.

Next, why bother with a bool and int for cool downs? Just use an int?, if it's null then there's no cool down, if its >0 then that's the cool down time. Pretty simple.

You use the term Command way too many times for my tastes.

internal interface ICommand
{
    void Process(Player player);
    string PermissionRequired { get; }
    int NumberOfParameters { get; }
    string Parameters { get; }
    string Description { get; }
    int? CooldownMs { get; }
}

Even then, NumberOfParameters is truly unnecessary. I'd bet there is a separator of some sort between commands in the Parameters string, which really shouldn't be a string, but should be something like the following:

public class CommandParameter
{
    public string Name { get; set; }
    public Type Type { get; set; }
    public string Description { get; set; }
}

Then Parameters becomes an IEnumerable<CommandParameter> which means you can use Parameters.Count in place of NumberOfParameters. This also means you can extrapolate further and define an abstract class for this, and add a CanProcess(Player player) method:

public abstract class BaseCommand : ICommand
{
    private List<CommandParameter> _parameters = new List<CommandParameter>();
    public IEnumerable<CommandParameter> Parameters => _parameters;

    public int? Cooldown { get; protected set; }
    public string Description { get; protected set; }
    public string PermissionRequired { get; protected set; }

    public abstract void ProcessCommand(Player player);

    public bool CanProcess(Player player)
    {
        // Check cool down, permissions, ban list, etc.
    }
}

Then in an actual command:

public class InfoCommand : BaseCommand
{
    public InfoCommand()
    {
        Cooldown = 5000;
        Description = "Lets you view information about the server.";
    }

    public override void ProcessCommand(Player player)
    {
        if (!CanProcess(player))
        {
            // I'm throwing an exception, but you could just `return` and send a message or whatever
            throw new InvalidOperationException("You have either exceeded the cool down, do not have permissions, or are banned. Please try again later.");
        }

        // Process our command here
    }
}

Then you created your own system that requires boilerplate code, and extracted that boilerplate code away.


Reading and writing off the end!

When it points to the last element (one before end), you read one-past-the-end, and then potentially write to it. This is undefined behavior. You need to make sure that you stop before then. One way to ensure this is to iterate from begin+1 to end, and compare the element with the one before it.

Logic

The typical way to write bubble sort is to have a loop that goes the full list, and set a flag if you swapped anything, and loop until you didn't. This will make it easier to understand what's going on - rather than having your loop next step set in two separate places, which is error prone.

Unnecessary variables

You don't need num1 or num2. Simply rely on std::swap:

if (*(it-1) > *it) {
    std::swap(*(it-1), *it);
    swapped = true;
}

Or you could implement such a thing yourself:

void swap(int& a, int& b) {
    int tmp = a;
    a = b;
    b = tmp;
}

Either way, avoiding unnecessary variables is a plus.

Spacing

Don't add so many blank statements between lines. Taking up too much vertical space makes it harder to read.

Proposed implementation

The following addresses all of my points:

void sort2(int* begin, int* end) {
    bool swapped = true;
    while (swapped) {
        swapped = false;
        for (int *it = begin+1; it != end; ++it) {
            if (*(it - 1) > *it) {
                std::swap(*(it - 1), *it);
                swapped = true;
            }
        }
    }
}

Minor optimization

Rewriting the way I did it above allows for a minor optimization. Every time through the for loop, we know that we just put the "largest" number at the end. It "bubbled" up! At that point, we don't need to do anything else with it, so we can decrement the end pointer:

while (swapped) {
    swapped = false;
    for (int *it = begin+1; it != end; ++it) {
       ...
    }

    --end; // <==
}

Future work

Bubble sort is \$O(n^2)\$. It gets the job done, but it's... not great. A strictly better algorithm to start with is insertion sort, which is still \$O(n^2)\$. From there, you can look at merge sort and quicksort, both \$O(n \lg n)\$.

Also consider what you'd need to do to be able to support (a) arbitrary types, not just ints and (b) in arbitrary order, not just increasing.


Converting phone numbers to words with Python

Currently you are checking all possible combinations of letters and see if it is in your dictionary. It is easier to convert all dictionary words (of the right length) into phone numbers and see if each matches your number. This is especially true if you want to check multiple phone numbers (you only need to do this conversion once). In fact, you can completely pre-compute this by using a dictionary mapping from phone numbers to all (four letter) words in the dictionary.

import pandas as pd
from collections import defaultdict

INVERSE_PHONE_DICT = {"a": "2", "b": "2", "c": "2", "d": "3", ...}

def parse_dictionary(file_name):
    words_df = pd.read_csv(file_name, encoding='latin1')
    words = words_df[words_df.WORDS.str.len() == 4].WORDS.str.lower()

    word_numbers = defaultdict(set)
    for word in words:
        numbers = "".join(INVERSE_PHONE_DICT[c] for c in word)
        word_numbers[numbers].add(word)
    return word_numbers

if __name__ == "__main__":
    file_name = r'C:\Python\Scripts\Data_Playground\unix_words.csv'
    word_numbers = parse_dictionary(file_name)

    phone_number = '0000009376'
    for x in word_numbers.get(phone_number[-4:], []):
        print(f"{phone_number[:3]} - {phone_number[3:6]} - {x.upper()}")

Note that this does not deal with words of length other than four.

I also used collections.defaultdict for easy building of the mapping, used the fact that pandas.Series are directly iterable, used an f-string to simplify the string parsing at the end, completely removed the generate_words function, ensured that the words are of the right length and lower case, added a if __name__ == "__main__": guard to allow importing from this script and follower Python's official style-guide, PEP8, regarding spaces after , in argument lists.


class FIR has nothing private. Should it be struct?

The line
  std::list< T > taps;  // Array of is are shifting each tick

is somewhat contradictory. It taps a list or an array?

I don't see how order is used. Does it need to be a class member at all?

operator()(T x) is very confusing. I would argue that it shall not be an operator at all; if you really want this to be an operator, consider operator<<. A metaphor of shifting into a filter is much more understandable than calling a filter.

// Can we do better here? Yes.


How to reduce number of hierarchical 'if statements'

In general, you can try inverting the condition on your outermost if block, and work your way in.  This will often result in less nesting, eg:

if condition1
  if condition2
    #ifblock
  else
    #elseblock2
  end
else
  #elseblock
end

Inside a loop, as you have your code, it would become this strcture, which is less nested:

if !condition
  #elseblock
  next
end
if !condition2
  #elseblock2
  next
end
#ifblock


Here's a long list of feedback. Sorry for the blunt tone, but I'm trying to be brief:

parsestr mutates a global variable but returns a value that's relevant to
that variable (the count). That's inconsistent. Either have it return both
the count and the array (or make things easier on yourself and use a
vector that knows its size), or make the count global too.
You don't handle punctuation characters. Is that a requirement? Should
"a(b)" be one word? Should "a. a" be two unique words?
Do you need to split on characters other than space? What about \t or
\n? Other Unicode whitespace characters?
Your while(i < maxlength) loop could more simply be a for loop.
Building up the word by incrementally contatenating characters onto a
string is slow. It will periodically require dynamic allocation. A more
efficient solution would be to remember the start index of the word. When
you reach the end, create a string from the substring (start, end) in one
step.

Going further from there, there's no reason to even store those words
separately since that start, end pair is enough to reconstruct it as needed.
Your program will mysteriously crash if you pass a sentence with more than
ten words. :( Since you're using dynamic allocation everywhere else (string
does that under the hood) there's no reason to use a fixed array for words.
At the very least, you should have a constant for that size and check that
you don't overflow the array.
index isn't a helpful variable name. Call it wordIndex.
Likewise, wordcnt would be better as wordCount.
maxrepeatedWord is a mixture of camelCase and all lowercase. Be consistent
(and camelCase is generally better since it makes it easier for the reader
to split it into words).
wordcntArr would be better as wordCounts. The fact that it's an array is
self-evident.
i <= count should be just <. Arrays are zero-based, so
wordcntArr[count] is past the last valid element.
index -> indexOfMax.
You're passing count into countwords even though that count is relevant
to a global variable that it accesses directly. Why not just make the count
global too?
The else {} accomplishes nothing. Remove it.
If you declare wordcnt inside the for(int i... loop, you won't have to
reinitialize it each iteration.
int wordcntArr[10] again duplicates the magical literal 10. Use a
constant or, better, a dynamically-sized container.
You're redundantly recounting each word every time it occurs. With
"I am am am good good", you'll get a count array like 1 3 3 3 2 2.
If you had a collection of unique words instead, your \$O(n^2)\$ complexity
would go down to \$O(mn)\$ where \$m\$ is the number of unique words.

A hash table would be a good way to create the set of unique words.
Your test string assumes repeated words will be contiguous (as opposed to,
say "I am good am good am."). Is that intentional? Desirable?

At a high level, your algorithm is also not optimal. You've got \$O(n^2)\$ performance, ignoring the dynamic allocation as you incrementally build up those strings. With that, it gets worse.

I believe the canonical solution to this is (roughly):

create a map of words -> counts
set wordStart to 0
iterate through the string
    if the current character is a word delimiter
        set word to the substring from wordStart to here
        set wordStart to here
        if map contains word
            increment the count for that word
        else
            add the word to the map with a count of 1
        end
    end
end

set maxWord to null
set max to 0
iterate through the map
    if count for this word > max
        set maxWord to this word
        set max to count
    end
end

return maxWord

That gives you \$O(n)\$: you only walk the entire string once.


The while loop can be written as follows (although I think it does not speed things up):

while True:
    # Determine the iterator that the next element should be taken from
    if key:
        low,high = (0,1) if key(tops[0]) <= key(tops[1]) else (1,0)
    else:
        low,high = (0,1) if tops[0] <= tops[1] else (1,0)
    nl.append(tops[low])

    # Update the top of the iterator
    tops[low] = next(its[low], None)
    if not tops[low]:
        # Unless the iterator is empty, in which case get the rest of
        # the values from the other iterator
        nl.append(tops[high])
        nl.extend(its[high])

        return nl


Testing a template function in GoLang using mocking - making it DRY & easy to follow

I don't know exactly what your fields type is, but from the code you've shared, there's a couple of strong recommendations I'd like to make about the code itself, before getting in to the testing aspect.
post, err := f.store.Posts.GetById(id[0])

From this line, and the fact that you're mocking store, I'm deducing your fields constructor looks something like this:
func New(store SomeStoreT) *fields {
    return &fields{
        store: store,
    }
}

Because you're accessing store.Posts directly, the SomeStoreT is going to be the actual type, complete with exposed fields. This means you can't really mock the store type as a whole, you have to ensure that whatever the exported fields are, is safe for concurrent use, and that no other code either on purpose or accidentally assigns anything to said fields. An extreme example would be:
func main () {
    store := storage.New()
    f := fields.New(store)
    store.Posts = nil
    f.GetPosts() // panic
}

It's therefore better to either replace the store dependency with an interface that exposes what you need through getters, so the fields needn't be exported. It'd be as simple as changing the line
post, err := f.store.Posts.GetById(id[0])

to:
post, err := f.store.Posts().GetById(id[0])

Still, there's something that I find a lot more disturbing than this: the use of interface{} and the json.Unmarshal() calls. Your store field has a Posts field. That tells me you know what data you're fetching, or saving. Why would you let code other than the storage layer deal with the marshalling and unmarshalling of that data? Your code should pass around a type that represents a post, and it's the job of the storage layer to handle the marshalling. If you decide JSON isn't the right format, and want to opt for something faster (like a binary format), or you want to switch to a completely different type of DB like Postgres, then you don't want to be forced to change the code throughout your application. You just should be able to swap out you storage adapter, optionally change the tags on the data types, and update the package to handle the new marshalling and be done with it:
package domain

type Post struct {
    ID      int    `json:"id"` // preferred ID in all-caps
    Content string `json:"content"`
}

Then, in the storage package:
package storage

type Posts struct {
    // whatever fields to access the storage used
}

func (p *Posts) GetByID(id int) (*domain.Post, error) {
   record, err := p.db.Get(id) // whatever you need to do here
   if err != nil {
      return nil, err
   }
   ret := domain.Post{}
   if err := json.Unmarshal(record, &ret); err != nil {
       return nil, err
   }
   return &ret, nil
}

So with this, you can change your function in fields to not have to do any unmarshalling at all. It's all handled in the storage. Now, getting a field by string isn't possible with this code, of course. I'm not sure of that's actually a good idea, but that's more of an X-Y problem. You can quite easily implement something that makes getting fields by string ridiculously easy, in fact. All you need to do is create a wrapper type and custom unmarshalling:
package domain

type Post struct {
    PostRecord // embedded struct
    postMap map[string]interface{}
}

type PostRecord struct {
    ID      int    `json:"id"`
    Content string `json:"content"`
}

func (p *Post) UnmarshalJSON(data []byte) error {
    // unmarshal into embedded type
    if err := json.Unmarshal(data, &p.PostRecord); err != nil {
        return err
    }
    if err := json.Unmarshal(data, &p.postMap); err != nil {
        return err
    }
    return nil
}

func (p Post) MarshalJSON() ([]byte, error) {
    return json.Marshal(p.PostRecord) // marshal the embedded type
}

func (p Post) GetField(f string) (interface{}, bool) {
    v, ok := p.postMap[f]
    return v, ok
}

Now your fields function would look something like this:
// you're only ever using `id[0]`, so I don't understand the variadic argument here
// it's also important to note that there's a lot of calls that can error happening here
// and you need to return those errors. It'll make debugging a lot easier
func (f *fields) getField(field string, id int) (interface{}, error) {
    post, err := f.store.Posts().GetByID(id)
    if err != nil {
        return nil, err
    }
    v, ok := post.GetField(field)
    if !ok {
        return nil, errors.New("unknown field for post type")
    }
    return v, nil
}

You'll note that I'm returning nil in case of an error. That's because an empty string is the nil value for a string, but it could just as well be a valid value for an existing record in some cases. It's best to indicate an error by returning said error, and nil for values that are just not defined. If a field doesn't exist, it has no value, and so nil is the only correct thing to return. Of course, the nil value for an int is 0, and an empty string for a string, but your function looks to me like it's written to be non-type-specific. If the Post type had a timestamp field, stored either as time.Time or int64, and you don't return an error, just an empty string, then the caller might do something like this:
tval := fields.getField("timestamp", 123)
ts, ok := tval.(int64)
if !ok {
    return errors.New("invalid timestamp value")
}

Now why is the timestamp value invalid? Becuase the field didn't exist (a typo in the string I passed)? Because the ID doesn't exist? Or because the timestamp field has since been updated to another type? I don't know...
This also assumes that the caller has the sense to check whether the type assertion worked. There's no reason why the caller might just have:
ts := tval.(int64)
and continue to use ts assuming it was the correct value, while in reality it'll default to 0. Write your functions to account for sloppy use. Errors are more likely to be checked than boolean indicators. Return values should be unequivocal when it comes to distinguishing between an error and success. An emtpy string is a perfectly valid return value on success, so don't rely on it to communicate failure.

I'll update this review gradually. I think this is a reasonable amount of information to digest, and get to work with. I've written all of the code snippets above off the cuff, untested, etc... there may be typo's and omissions, so don't assume they're copy-paste ready.

TL;DR answers to your questions

You have to mock dependencies in unit tests. that's what unit tests are/do
I've started this review with some suggestions/recommendations to improve on your code - so yes, there are things that you can do there
with the changes I've proposed, the getFields function is basically a call to the storage package (which is tested separately), and a call on the domain object's GetField function, which is something you unit-test separately. you can still write tests for the function here, but it's just having a mock return either an error or a Post instantce, which you can hard-code in your test. It's far less cumbersome to set up, and test as a result.


Network programming - asynchronous, single-threaded, nickname-only chat server

Packet fragmentation

Have a read through this: https://stackoverflow.com/a/17668009/313768

usock.recv(1024) is fragile, and under many circumstances simply won't work for a handful of reasons. You'll need to do some reading about the fundamentals of TCP/IP to understand why that is, but basically, you can't assume that you'll get the data you want all in one chunk.

Context manager

You hold onto some sockets, which are resources that should be closed when you're done with them even if something goes wrong. This is a job for a context manager. You can make your class a context manager so that calling code can be sure that your resources are closed off, regardless of what else happens.

Have a read through this: https://docs.python.org/3/library/stdtypes.html#typecontextmanager

Presentation versus logic

    usock.send(b"Please, choose your username: ")

Typically, this is not done, and instead a pre-chosen number is sent over the wire, leaving the client to choose how to ask the user a particular thing. This becomes especially important when graphics get involved, or when you need to do translation/internationalization.

The first thing

[u for u in self.users if u.sock is u.sock][0].name

should be

next(u for u in self.users if u.sock is u.sock).name

That said... what on earth is this doing? When would that condition ever evaluate to False?

Type hints

PEP484 type hints, such as host: str, port: int, will clarify your function arguments.


Computational verification of Collatz conjecture using gmplib

If you need a type with exactly 64 bits, use uint64_t (from <stdint.h>).  That will give a clear compilation error if no such type is available.

Given that the function never returns anything other than zero, we can move the cleanup of n, n0 and n_max outside the loop, and simply break to reach them.

Is indefinite looping really a good output if a cycle is detected?  How is that distinguishable from an arbitrarily long (but finite) chain?  Look up the standard algorithms for cycle detection in your graph theory textbook.