Enabling discard pending changes on DbContext

As stated by previous comments, there is no need to throw away the context.
But since this is a code review, and not a concept review I'll leave that part aside.

All your TContext generics are to be inheriting from DbContext, but nowhere in any class do you use any of DbContext's properties or methods. You are only using the .Dispose method, which comes from the IDisposable interface. I've refactored your code to have this result:

public interface IContextFactory<out TContext> where TContext : IDisposable
{
    TContext Create();
}

public interface IDiscardModelChanges
{
    void DiscardChanges();
}

public abstract class DiscardableModelBase<TContext> : IDiscardModelChanges, IDisposable
where TContext : IDisposable
{
    private readonly IContextFactory<TContext> _factory;

    protected TContext Context { get; private set; }

    protected DiscardableModelBase(IContextFactory<TContext> factory)
    {
        _factory = factory;
        Context = _factory.Create();
    }

    public virtual void DiscardChanges()
    {
        Context.Dispose();
        Context = _factory.Create();
    }

    public void Dispose()
    {
        Context.Dispose();
    }
}

The changes might seem small, but the impact is big. In your example, you are going to pull in Entity Framework everywhere you are using this application, which isn't needed at all.
You are not doing anything entity framework related, you are doing something with disposable objects. This could be a StreamWriter, etc...

PS: I know this post is old. But people should really be aware of the issue with this code.


Split large file into smaller files, keeping headers and footers

When writing lots of data, it's better to write all of it in one shot wherever possible.

My code only addresses your specific performance issue, e.g. takes too long to run! It lacks the header and footer manipulation needed, however this should be easy to add in at this point. I'd recommend manipulating this in an Array or StringBuilder before ready to write. Overall, your code is clear, just the method employed is non-optimized for this much data.

I separated the directory loop from the split action. It seemed to me finding a qualifying file and splitting the file were separate actions, so it made sense (to me anyway) to split these activities up into separate Subs.

I used utilized the FileSystemObject to make it easier to build a new file path for new files that were split. Also, you had some Magic Numbers I made those into constants, as it is, it's non-obvious what 1000000 is meant to represent.

Feel free to ask questions if something isn't clear, I tried to comment the code as much as possible.

Edit

Made a few tweaks changes with how I'm reading the file in. Seems to be about 10% faster.

Edit 2

For completeness I included (stole?) Thomas Inzina great idea for reading the file line by line. This sped things up considerably. See the revised timings below.

Code

Option Explicit

Public Sub FindFilesToSplit()

    Dim FolderPath As String
    Dim FileNames  As String
    Const FileSizeLimitBytes As Long = 20000000
    FolderPath = "E:\Ex\"

    FileNames = Dir(FolderPath)

    Do While Len(FileNames) > 0
        If (FileLen(FolderPath & FileNames) / FileSizeLimitBytes) > 1 Then SplitFiles (FolderPath & FileNames)
        FileNames = Dir
    Loop

End Sub

Private Sub SplitFiles(ByRef FilePath As String)
    Const BytesToMBs As Long = 1000000
    Const FileSizeThresholdMBs As Long = 20

    Dim TimeRoutine     As Single: TimeRoutine = Timer
    Dim FSO             As FileSystemObject
    Dim FileNumber      As Long
    Dim FileSize        As Long
    Dim FileData()      As String
    Dim NumberOfFiles   As Long
    Dim LinesToRead     As Long
    Dim i               As Long
    Dim j               As Long
    Dim k               As Long
    Dim SplitFileName   As String
    Dim StartingLine    As Long
    Dim EndingLine      As Long
    Dim ChunkofFile     As Variant

    Set FSO = New FileSystemObject
    FileSize = FileLen(FilePath) \ BytesToMBs

    'Compute how many files are going to be needed
    NumberOfFiles = (FileSize \ FileSizeThresholdMBs) + 1

    'Using Thomas Inzina approach here instead, it's a lot faster
    'Awesome stuff Thomas Inzina :)
    FileData = GetTextFileLines(FilePath)

    Debug.Print "Reading and splitting the file took: " & Timer - TimeRoutine & " seconds. The file size is: " & FileSize & " MBs"
    TimeRoutine = Timer

    'Compute the number of lines to read for each iteration
    LinesToRead = (UBound(FileData) \ NumberOfFiles) + 1

    'Use to variables to keep track which lines to read
    'Assumption: each line equally is equal terms of space requirements
    'To be safe you may want to increment LinesToRead by 2 just in case :)
    StartingLine = LBound(FileData)
    EndingLine = LinesToRead

    For i = 1 To NumberOfFiles
        'Resize an array to hold data for a single file
        ReDim ChunkofFile(0 To (EndingLine - StartingLine))

        k = 0
        'Add the text back to a smaller array
        For j = StartingLine To EndingLine
            ChunkofFile(k) = FileData(j)
            k = k + 1
        Next

        'Build the string name for the new file
        'The file name mirrors the parent file, just with an
        'iteration number suffix
        SplitFileName = FSO.GetParentFolderName(FilePath) & "\" & _
                        FSO.GetBaseName(FilePath) & "_" & CStr(i) & _
                        "." & FSO.GetExtensionName(FilePath)

        StartingLine = StartingLine + LinesToRead + 1

        'Adjust ending line to read as dividing by the number of files -
        'won't divide evenly (most of the time)
        If i = NumberOfFiles - 1 Then
            EndingLine = UBound(FileData)
        Else
            EndingLine = EndingLine + LinesToRead + 1
        End If

        'Write the file by joining the array just created
        FileNumber = FreeFile()
        Open SplitFileName For Output Access Write As FileNumber
        Print #FileNumber, Join(ChunkofFile, vbNewLine)
        Close #FileNumber
    Next

    Debug.Print "Rest of Process took: " & Timer - TimeRoutine & " seconds"
End Sub

Private Function GetTextFileLines(ByRef Path As String) As String()
    Const MAX_ROWS  As Long = 30000000
    Dim TextLine    As String
    Dim FileNumber  As Integer
    Dim i           As Long
    Dim TextArray() As String

    ReDim TextArray(MAX_ROWS)

    FileNumber = FreeFile
    Open Path For Input Access Read As #FileNumber

    Do While Not EOF(FileNumber)
        Line Input #FileNumber, TextLine
        TextArray(i) = TextLine
        i = i + 1
    Loop
    Close #FileNumber

    ReDim Preserve TextArray(i - 1)
    GetTextFileLines = TextArray
End Function

Results

The performance is pretty good. I'm processing a ~200MB file into 11 files in about 12 seconds.

Debug Messages:

Reading and splitting the file took: 4.609375 seconds. The file size is: 219 MBs
Rest of Process took: 7.953125 seconds

Edit 3

Let's go for even faster performance! It's been fun trying to make this even faster.

Splitting really slows things down, so I'm ditching it entirely. Instead I'm writing every N number of characters instead. So you do lose the tidy look of split per line with this approach, but the files are still split up equally. It should be possible to keep the lines together by looking ahead/behind to find the linebreak character, I just didn't bother.

I'm using several Win APIs to get a speed boost in a few places. To keep this simple, I did not add the VBA7/win64 API signatures, but this should be easy to do.

Improvements

The Space$() buffer allocation can be somewhat slow when allocating that much buffer, so I replaced with a new approach.

Also, I'm using the CreateFile API too. VBA is a kinda slow writing files, this helped a bit with speed too. All in all, I've got splitting and writing a 130MB file in less than 2 seconds (~1.89 seconds).

Revised Code

Option Explicit

Private Const GENERIC_WRITE = &H40000000
Private Const GENERIC_READ = &H80000000
Private Const FILE_ATTRIBUTE_NORMAL = &H80
Private Const OPEN_ALWAYS = 4
Private Const INVALID_HANDLE_VALUE = -1
Private Const CREATE_ALWAYS = 2
Private Const BytesToMBs As Long = 1000000
Private Const FileSizeThresholdMBs As Long = 20
Private Const FileSizeLimitBytes As Long = 20000000
Private Const FILE_FLAG_WRITE_THROUGH = &H80000000

Private Declare Sub RtlMoveMemory Lib "kernel32" (dst As Any, src As Any, ByVal nBytes As Long)
Private Declare Function SysAllocStringByteLen Lib "oleaut32" (ByVal olestr As Long, ByVal BLen As Long) As Long

Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long

Private Declare Function WriteFile Lib "kernel32" (ByVal hFile As Long, lpBuffer As Any, _
    ByVal nNumberOfBytesToWrite As Long, _
    lpNumberOfBytesWritten As Long, ByVal lpOverlapped As Long) As Long

Private Declare Function CreateFile Lib "kernel32" Alias "CreateFileA" (ByVal lpFileName As String, _
    ByVal dwDesiredAccess As Long, _
    ByVal dwShareMode As Long, _
    ByVal lpSecurityAttributes As Long, _
    ByVal dwCreationDisposition As Long, _
    ByVal dwFlagsAndAttributes As Long, _
    ByVal hTemplateFile As Long) As Long

Private Declare Function FlushFileBuffers Lib "kernel32" (ByVal hFile As Long) As Long

Private Function AllocString(ByVal Size As Long) As String
    RtlMoveMemory ByVal VarPtr(AllocString), SysAllocStringByteLen(0, Size + Size), 4
End Function

Public Sub FindFilesToSplit()
On Error GoTo ErrorHandler:

    Dim FolderPath As String
    Dim FileNames  As String

    FolderPath = "E:\Ex\"
    FileNames = Dir$(FolderPath)

    Do While Len(FileNames) > 0
        If (FileLen(FolderPath & FileNames) / FileSizeLimitBytes) >= 1 Then SplitFiles (FolderPath & FileNames)
        FileNames = Dir$
    Loop

CleanExit:
    Exit Sub

ErrorHandler:
    Resume CleanExit
End Sub

Private Sub SplitFiles(ByRef FilePath As String)
    Dim TimeRoutine     As Single: TimeRoutine = Timer
    Static FSO          As FileSystemObject
    Dim FileNumber      As Long
    Dim FileData        As String
    Dim NumberOfFiles   As Long
    Dim CharsToRead     As Long
    Dim i               As Long
    Dim SplitFileName   As String
    Dim StartingChar    As Long
    Dim EndingChar      As Long

    If FSO Is Nothing Then Set FSO = New FileSystemObject

    'Compute how many files are going to be needed
    NumberOfFiles = ((FileLen(FilePath) \ BytesToMBs) \ FileSizeThresholdMBs) + 1

    'Get File data
    TimeRoutine = Timer
    FileNumber = FreeFile()
    Open FilePath For Binary Access Read As FileNumber
    FileData = AllocString(LOF(FileNumber))
    Get FileNumber, , FileData
    Close FileNumber

    Debug.Print "Reading and splitting the file took: " & Timer - TimeRoutine & " seconds. The file is 130 mb"
    TimeRoutine = Timer

    'Compute the number of lines to read for each iteration
    CharsToRead = (Len(FileData) \ NumberOfFiles)

    'Counters to keep track of which character we read
    StartingChar = 1
    EndingChar = CharsToRead

    For i = 1 To NumberOfFiles
        'Create the file name
        With FSO
            SplitFileName = .GetParentFolderName(FilePath) & "\" & _
                            .GetBaseName(FilePath) & "_" & CStr(i) & _
                           "." & .GetExtensionName(FilePath)
        End With

        'Write the file
        WriteStringToFile SplitFileName, Mid$(FileData, StartingChar, CharsToRead)
        StartingChar = StartingChar + CharsToRead + 1

        'Adjust ending line to read as dividing by the number of files -
        'won't divide evenly (most of the time)
        EndingChar = IIf(i = NumberOfFiles - 1, Len(FileData), EndingChar + CharsToRead + 1)
    Next

    Debug.Print "Rest of Process took: " & Timer - TimeRoutine & " seconds"
End Sub

Private Sub WriteStringToFile(ByRef FileName As String, _
                              ByRef FileData As String, _
                              Optional NoOverwrite As Boolean = False)

    Dim FileHandle         As Long
    Dim Success            As Long
    Dim BytesWritten       As Long
    Dim BytesToWrite       As Long

    'Quick Fail
    If NoOverwrite = True And Dir$(FileName) <> vbNullString Then Exit Sub

    BytesToWrite = Len(FileData)

    FileHandle = CreateFile(FileName, GENERIC_WRITE Or GENERIC_READ, _
                            0, 0, CREATE_ALWAYS, FILE_FLAG_WRITE_THROUGH, 0)

    If FileHandle <> INVALID_HANDLE_VALUE Then

       Success = WriteFile(FileHandle, ByVal FileData, BytesToWrite, BytesWritten, 0) <> 0

       If Success <> 0 Then
          Success = FlushFileBuffers(FileHandle)
          Success = CloseHandle(FileHandle)
       End If

    End If

End Sub


Some notes:

This is not a unit test of your code. A unit test must at some point involve some part of your code, and it looks like this test uses only classes which exist in the Java SDK. The idea is not to check an idea for an implementation, but instead to detect bugs in your actual implementation.
By convention instance variables use camelCase in Java, so ExpectedResult should be expectedResult and str_time2 should be strTime2.
Maybe it's necessary, but I'm sceptical of the need for the conversion from Date to DateTime.


Limit HTTP request rate for spray/akka-http

I think you could rewrite your limit function like this :

def limit[T](requestRateLimiter: RequestRateLimiter[T]) = {
  def remoteAddress = optionalHeaderValuePF { case `Remote-Address`(address) ⇒ address }
  def xForwardedFor = optionalHeaderValuePF { case `X-Forwarded-For`(Seq(address, _*)) ⇒ address }

  for {
    ctx <- extract(identity)
    realIp <- remoteAddress
    proxiedIp <- xForwardedFor
  } yield requestRateLimiter.check(ctx, realIp, proxiedIp) match {
    case Pass(result) => provide(result)
    case Block(rejection) => reject(rejection)
  }
}

Here are the two main changes that I've done :

Changed your flatMaps for a "for comprehension" (which imho is more readable)
Removed the types on the declaration of the two variables.

Overall, I find it more readable.


No and no. This is a very inefficient way to reverse, for example, a dynamic array based list (it's a different story with a linked list). With a dynamic array based (e.g. ArrayList) The remove(0) operation will require moving every other element in the list (linear in the length), so the whole method is quadratic in the total length of the list. It also requires a linear number of stack frames, which are actually pretty expensive and if you ask for too many the program will crash, which means this method will not work for long lists.

An efficient solution would work in linear time with a constant space requirement.

When thinking about inputs, it's important to also consider null as a possible input. The deficient spec does not indicate what should be done with a null input, which means you have had to make a decision, and that decision should be documented. The 'easy' (and often best) option is to explicitly test for a null input and throw an IllegalArgumentException exception, so that anyone using your method receives a useful error message. You could argue that your code should do nothing with a null, but that's a bit iffy, because the spec doesn't qualify the behaviour, and that would 'silently' fail if someone passed it a null incorrectly: throwing an exception forces any unexpected usage to be address directly, which may or may not prompt a decision change, which makes it a good 'default' choice.

There is no need here to pre-declare t: just declare and assign it in one go:

T t = list.get(0);

This makes it easier to find out what it means. You could also make it final in this case, so that it is entirely clear that it is only assigned once, which makes it that little bit easier to reason about the code.

When writing recursive code like this, it can pay select the 'most ideal' base case. In this instance, any list of length no more than 1 is already reversed, so you could change the if condition to list.size() > 1. This doesn't change the complexity, but is something of which to be aware.


Return target informations from nested api json output

1. Getting data from dict
When details is a dict, then instead of writing:
try:
    address = details['result']['formatted_address']
except KeyError:
    address = ""

you can do:
address = details.get('result', {}).get('formatted_address', '')

Second parameter of .get represents the default value which is returned when a dictionary doesn't contain a specific element
2. Modularization
Function pipeline_place_details is not short, so it might be a good idea to break it up into a smaller functions. Each time when you write a loop, it's worth to consider if moving body of the loop to a separate function will increase the code readability.
For example that part:
    author_name = review['author_name']
    user_rating = review['rating']
    text = review['text']
    time = review['relative_time_description']
    rev_temp.append((author_name, user_rating, text, time))

can be easiliy extracted to the new function:
def process_review(review):
        return (
            review['author_name'],
            review['rating'],
            review['text'],
            review['relative_time_description']
        )

and you can use it as below:
rev_temp = [process_review(review) for review in reviews]

3. Variables naming
Good code should be not only working and efficient but it should be also easy to read and understand. So it's a good rule to use really meaningful variable names. I belive that you can find better names than for example rev_temp or rev_temp2. ;)


Sourcing content from a webpage in an organized manner

Perhaps I misunderstand you. But it looks as though you've already written the functions to call one another: get_info() calls make_post() internally, and make_post() then delegates getting the token to get_token() by calling it internally to itself. So in your final line of code, you should be able to replace the three calls with a single call to get_info().

Whether or not that's "better" than calling them independently is a question of style. Personally, I think it's best to keep the functions independent and call them separately. This would mean re-writing them to take parameters that are the results of the preceding function call.

This way the functions don't have to know about each other's existence, and means you will be able to change each one independently over time as requirements evolve, without needing to update the others (this is called low coupling, a very useful design quality to keep in mind).


Why Encryption Doesn't Fit the Goal of Password Storage

Cryptographic hashing and encryption have two different purposes. Hashing is not reversible -- given a hash, you can't determine what made it except by testing to see if a source input matches the hash output. When used for passwords, this becomes a validation only question: "does this password (mixed with this salt) match?" With passwords, the goal is to prevent somebody with full access to the database from recovering passwords.

Encrypting passwords handles that goal poorly. To verify a password, one must have the encryption key. That means one is also able to determine the source password from looking at the encrypted version of the password. In the event of a compromise, having those encrypted fields and the key will result in the disclosure of all passwords regardless of how strong they were.

Good Resources

https://security.stackexchange.com/questions/211/how-to-securely-hash-passwords

The Rules of Crypto

You'll hear it a lot around here: don't roll your own. The problem is that the field has changed too much for "common sense" to really be something that's working on our favor. Encryption and password issues have evolved relatively fast on a human-life scale of time. Where not everybody is following that, a lot of varying ideas exist about how to solve these problems and many of them have been proven wrong.

If you're not current, the proper approach is to research the issue you're trying to solve rather than the method you're trying to solve it with.


Python has a style guide, PEP8. I recommend that you follow it, as it makes your code easier to read.
In this case it's mostly just using snake_case, rather than camelCase.
You should use with to automatically close the file. This happens in any circumstance.
Rather than using if startTag in bearFile, I'd invert the if and continue.
Rather than using while you can use for still. This, IMO, is easier to understand too.
You could use a generator function so you don't have to write as much.

Along with a couple of name edits, here's how I'd change your code:

def parse(path, start, end):
    with open(path) as file:
        for line in file:
            if start not in line:
                continue

            for line in file:
                if end in line:
                    break
                yield line.strip()

print(list(parse("bears.txt", "==start==", "==end==")))

To address your concerns:

Do this out of the function. If you ever do want these empty values, then you'll be duplicating code. You also only need to use:

[item for item in parse(...) if item]

You could implement a function, split_delim, that splits on a delimiter. If you make the function simple, then all you'd need to do is add one line to your function:

def parse(path, start, end):
    with open(path) as file:
        file = split_delim(file, '\n')
        for line in file:
            ...

My changes should remove this error, due to how Python's for loops work. However I havn't tested this.


Using Powershell to Copy Files to Several Locations

Simple program at first glance. I am glad to see some documentation there that explains what the code does. This is a good and important practice.

Set-Location

This is the only real problem I see with the code here. You rely on the working directory of PowerShell when copying files. You already know the directories you are using. Use the full paths and skip Set-Location altogether. This will reduce complexity and points of failure.

Instead of

# copy to the MM Client folder
$destination = ($codeToDeploy + $datedFolderName + "OurProjectName Client\");
Set-Location $myLocalOurProjectName
Copy-Item *.dll,OurProjectName.exe $destination -Force;

Use

# copy to the MM Client folder
$destination = ($codeToDeploy + $datedFolderName + "OurProjectName Client\");
Copy-Item -Path $myLocalOurProjectName -Include "*.dll","OurProjectName.exe" -Destination $destination -Force

In this same vein use the parameter names when writing code. Position arguments are fine for testing but readability improves for you and others when you use named arguments.

New-Item

When creating new folders PowerShell will output the Directory Object when it is created. Most find this undesirable. To void the output a clean approach is to pipe Out-Null. There are other ways to do this as well.

New-Item $destination -ItemType directory -Force | Out-Null

You might like the output but just beware it is there and being generated by New-Item.

String based path creation

This can work fine but a better practice is to use cmdlets and methods to build paths reliably. This way you don't have to worry about leading and trailing as much. When it comes to stitching more that 2 folder/paths together Combine is your friend.

Instead of

$destination = ($codeToDeploy + $datedFolderName + "OurProjectName Client\");

Consider

[io.path]::Combine($codeToDeploy, $datedFolderName, "OurProjectName Client\")

Code Repetition

This is largely made moot by what was said previously but keep this in mind. You have small code blocks that are repetitions but use the same logic with just some strings changing. Functions should be used here so that if some logic has to be added you are making the change in one place as supposed to updating several snippets throughout the code.

Pointless Code

Understand that I really wanted another word other than pointless. You have at least one line that is not serving a purpose

$folders = Get-ChildItem "\\nw\data\TnFS\Project\ProjectFolder\UI TESTING\MM *" | Sort-Object

There was commented code related to the above line but, since it was commented, the above line serves no purpose. Something to watch out for.


Grouping all of the view components into a single view can make views easier to reuse, which can be useful in some circumstances.

However, it is not necessary to create a separate view for each view controller, and view controllers often have individual properties for the subview.

While there is nothing technically wrong with your code, it does add an extra burden of having to define a view class for each view controller.

It is also not the "standard" approach, so it might seems out of place to other developers working on the code, which may or may not be a factor to you.

Usually the views are kept as simple as possible, with the individual controls defined on the view controller instead of the view. The view controller becomes the point of reuse instead of the view. View controller composition can be used to create a hierarchy of views.

Most of the code you have shown in the view can be moved into the view controller, which will remove the need to define a class for each view for each view controller.

When in doubt, aim for code that looks the least unusual, or even code that looks boring or obvious.

For example, instead of creating a OneWayView, create a OneWayViewController which contains the same logic as the previous view and view controller.

class OneWayViewController: UIViewController {

    // Probably better to keep this as a weak property.
    let myButton: UIButton = {
        let button = UIButton(type: .custom)
        button.translatesAutoresizingMaskIntoConstraints = false
        button.setTitle(NSLocalizedString("myButton", comment: ""), for: .normal)
        button.setTitleColor(.orange, for: .normal)
        button.setTitleColor(.lightGray, for: .highlighted)
        return button
    }()

    override func viewDidLoad() {
        super.viewDidLoad()

        // Add the button to the root view, which is just a plain view.
        view.addSubview(myButton)

        // Setup constraints.
        myButton.centerXAnchor.constraint(equalTo: centerXAnchor).isActive = true
        myButton.centerYAnchor.constraint(equalTo: centerYAnchor).isActive = true

        // Add target actions.
        myButton.addTarget(self, action: #selector(myButtonTapped), for: .touchUpInside)
    }
}

This new view controller can be composed with other view controllers:

class MainViewController: UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()

        let viewController = OneWayViewController()
        viewController.willMoveToParentViewController(self)
        addChildViewController(viewController)

        let oneWayView = viewController.view
        view.addSubview(oneWayView)

        // Add constraints to position the oneWayView...
    }
}

On the other hand, if you prefer to stick to your original plans, you could use Swift generics to make the view controller more reusable:

class BaseViewController<T: UIView>: UIViewController {

    var contentView: T {
        guard let contentView = self.view as? T else {
            fatalError("Cannot create content view")
        }
        return contentView
    }

    override func loadView() {
        self.view = T()
    }
}


Counting number of messages for a mailbox over several periods

Some suggestions ...

Use a counter cache for messages on Mailbox -- the count of all can be read from the mailbox instance.

Use #size instead of #length. This will use a counter cache for messages on Mailbox if one is present, or will use a SQL count. #length will get all of the messages back, then check the size of the array.

I would be concerned about having multiple mailboxes. If your user model has_many :messages, through: :mailbox you can run these queries more efficiently:

messages = current_user.messages

@today    = messages.today.size
@week     = messages.week.size
@month    = messages.month.size
@all_time = messages.size

With a counter cache, you're only running three queries to get the counts, and thy should be efficient enough.

Place a single index on the columns (user_id, id) on the mailboxes tables, and an index on (mailbox_id, created_at) on the messages table.

Edit: I just noticed the week and month variable assignments were round the wrong way. Also, if you wanted to be slightly more efficient you could define a scope for week_except_today and month_except_week, etc, and then:

@today    = messages.today.size
@week     = @today + messages.week_except_today.size
@month    = @week  + messages.month_except_week.size
@all_time = messages.size


Code smell when recursively returning a List

I wouldn't say there is a code smell with what you have now. However, it could be made more OOP-friendly.

An actual problem is that you're using raw types for Node, and that is never a good idea. So, first of all, consider having this instead:

private List<T> getInfix() {
    List<T> result = new ArrayList<>();
    getInfixInternal(root, result);
    return result;
}

private void getInfixInternal(Node<T> node, List<T> infixValues) {
    if (node != null) {
        getInfixInternal(node.getLeft(), infixValues);
        infixValues.add(node.getValue());
        getInfixInternal(node.getRight(), infixValues);
    }
}

and

public class Node<T extends Comparable<? super T>>

Then, to avoid this getInfixInternal, you can reason with what the method getInfix means. The first thing to realize is that it works on a given Node. In fact, it is really a property of the node itself: every node can return a list of values resulting of a infix traversal of the subtree starting at this node. Therefore, instead of having an internal method taking a Node as parameter, make it a method of Node:

public class Node<T extends Comparable<? super T>> {

    // ...

    public List<T> getInfix() {
        List<T> result = new ArrayList<>();
        if (leftChild != null) {
            result.addAll(leftChild.getInfix());
        }
        result.add(value);
        if (rightChild != null) {
            result.addAll(rightChild.getInfix());
        }
        return result;
    }

}

With such a method, you can then have:

private List<T> getInfix() {
    return root == null ? Collections.emptyList() : root.getInfix();
}

With Java 8 Optional, you could potentially write that more concisely:

public List<T> getInfix() {
    List<T> result = new ArrayList<>();
    Optional.ofNullable(leftChild).map(Node::getInfix).ifPresent(result::addAll);
    result.add(value);
    Optional.ofNullable(rightChild).map(Node::getInfix).ifPresent(result::addAll);
    return result;
}

Final note: you could also consider returning unmodifiable lists.


Square root approximation with Newton's method

Looks fairly good over all, but a few things jumped out at me. Please note that the stylistic ones are opinion based.

Make sure you don't get into a habit of using namespace std;. It's acceptable in some places, but it can form a bad habit very easily. Your code is a perfect example of this since your sqrt will conflict with std::sqrt if you include cmath or math.h. A lot more discussion on the matter can be found here.

If you're not going to put your function in a namespace, I would avoid calling it sqrt. In fact, unless you have a compelling reason, it's best to avoid names from the standard library in general.

I would leave a blank space after your header lines. The typical format is:

#include "local1.h"
#include "local2.h"

#include <blah1>
#include <blah2>

// stuff here

Also, I would consider alphabetizing the headers. When there's a long list of headers, it can help make quick mental scanning easier.

You forgot to include string.

Your c variable in sqrt isn't used.

I don't like the use of the constructor form initialization of primitives. I would stick with plain old equals.

Also, it's fairly typical to define one variable per line when values are being assigned during declaration.

double g = 1;
double ng;

I'm not quite sure why sqrt is templated. By using doubles internally, you've basically restricted it to doubles. If you're going to make it templated, use the templated type all the way through.

Y is a bad typename. Use something more descriptive like FloatingPoint or Number.

Consider taking x by const reference in sqrt. If an expensive-to-copy class is used as the template parameter, you're causing an unnecessary copy since x is never modified.

In menu, you should check if reading in a double was successful:

if (std::cin >> x) {
    // Use x
} else {
    std::cerr << "Invalid value provided\n";
}

Try to declare variables as close to usage as possible. For example, your g = sqrt(x); could be double g = sqrt(x);. Seeing a giant list of variables declared at the top of a function is overwhelming. If they're defined where they're used, it's much simpler to see their relation to the overall code.

If you're using C++11, avoiding naming types except where necessary. This will ease future change of type. For example, imagine that at some point you want to use some kind of Complex class instead of a built in double. If you only name the type for the input x and use auto else where, you'll only have to change one thing (provided the class overrode operator>> for istreams). You can read more about this here.

When it's not possible for the program to return a non-0 exit code, I like to omit it. This signifies at a simple glance that the program always exits with a success code.


The final memcpy may overflow output_buffer. I recommend to change the signature to

int str_contains(const char * str, char c, int *indices, size_t indices_size);

The caller shall compare the return value with the size of buffer it passed and act accordingly.
Dynamic memory is uncalled for. You may safely collect indices directly into the passed buffer.
There is no need for strlen.
When appropriate, use pointers.

All that said,

int str_contains(const char * str, char c, size_t * indices, size_t indices_size)
{
    size_t occurrences = 0;
    for (char * cursor = str; *cursor; cursor++) {
        if (*cursor == c) {
            if (occurrences < indices_size) {
                indices[occurrences] = cursor - str;
            }
            occurrences++;
        }
    }
    return occurrences;
}


determining if a range is within array of ranges

Just an idea

public struct Range
{
    public readonly int From;
    public readonly int To;

    public Range(int from, int to)
    {
        From = from;
        To = to;
    }

    public bool Contains(Range other)
    {
        return Contains(other.From, other.To);
    }

    public bool Contains(int from, int to)
    {
        return From <= from && To >= to;
    }
}

public class MyStruct
{
    public readonly Range Range1;
    public readonly Range Range2;

    public MyStruct(Range range1, Range range2)
    {
        Range1 = range1;
        Range2 = range2;
    }

    public MyStruct(int from1, int to1, int from2, int to2)
    {
        Range1 = new Range(from1, to1);
        Range2 = new Range(from2, to2);
    }

    public bool Contains(Range r)
    {
        return Range1.Contains(r) || Range2.Contains(r);
    }

    public bool Contains(int from, int to)
    {
        return Range1.Contains(from, to) || Range2.Contains(from, to);
    }
}

-

List<MyStruct> myStructs = new List<MyStruct>()
    {
        new MyStruct(86, 724, 725, 1364),
        new MyStruct(86, 724, 2068, 2707),
        new MyStruct(725, 1364, 2068, 2707)
    };

bool b = myStructs.Any(r => r.Contains(88, 100));


Filtering and adding additional properties to array of objects

It may be a better idea to use a reducer to initially add the properties to a subset of the elements of the array. Finally you can change the last element from the new Array with a property isLastLayer. I used Array.find within the reducer lambda and a custom function findLast for finding the last array-element with property isLastLayer.

const result = document.querySelector("#result");
const log = (...txt) => result.textContent += txt.join("\n") + "\n";

// moved to function, so the relevant code is more visible
const array = getTestArray();

// find the last element using the predicate lambda
const findLast = (array, predicate) => {
  for (let i = array.length - 1; i >= 0; --i) {
    if (predicate(array[i])) {
      return array[i];
    }
  }
};

// reduce the values, adding the layer props to
// all elements where the name does not start
// with 'bus' or 'truck'. Use Array.find within the
// accumulated array to determine if isFirstLayer
// is already in the set of elements
let newArray = array.reduce((acc, value) => {
  if (/^(bus|truck)/i.test(value.name)) {
    return [...acc, value];
  }
  const isFirst = !acc.find(el => "isFirstLayer" in el);
  value = { ...value,
    isFirstLayer: isFirst,
    isLastLayer: false
  };
  return [...acc, value];
}, []);

// use findLast to find the last element with
// a property 'isLastLayer' in the array, and
// change its value
findLast(newArray, el => "isLastLayer" in el).isLastLayer = true;

// let's see if it worked
log(JSON.stringify(newArray, null, 2));

function getTestArray() {
  return [{
      "name": "Truck",
      "status": "Cancelled",
      "id": 10
    },
    {
      "name": "Bus",
      "status": "Approved",
      "id": 11
    },
    {
      "name": "Car1",
      "status": "Approved",
      "id": 12
    },
    {
      "name": "Car2",
      "status": "Cancelled",
      "id": 19
    },
    {
      "name": "Car3",
      "status": "Cancelled",
      "id": 13
    }
  ];
}

It can also be done without the extra findLast function

const result = document.querySelector("#result");
const log = (...txt) => result.textContent += txt.join("\n") + "\n";
const array = getTestArray();
let accumulator = {
  newArr: [],
  // track first/last
  firstLayerSet: false,
  lastLastLayer: null
};
const addPropsReducer = (acc, value, i) => {
  if (/^(bus|truck)$/i.test(value.name)) {
    return {
      newArr: [...acc.newArr, value]
    };
  }
  value = { ...value,
    isFirstLayer: !acc.firstLayerSet,
    isLastLayer: false
  };
  return {
    ...acc,
    newArr: [...acc.newArr, value],
    firstLayerSet: true,
    lastLayerIndex: i
  };
};

let newArray = array.reduce(addPropsReducer, accumulator);
newArray.newArr[newArray.lastLayerIndex].isLastLayer = true;

// ditch tracking values
newArray = newArray.newArr;

log(JSON.stringify(newArray, null, 2));

function getTestArray() {
  return [{
      "name": "Truck",
      "status": "Cancelled",
      "id": 10
    },
    {
      "name": "Bus",
      "status": "Approved",
      "id": 11
    },
    {
      "name": "Car1",
      "status": "Approved",
      "id": 12
    },
    {
      "name": "Car2",
      "status": "Cancelled",
      "id": 19
    },
    {
      "name": "Car3",
      "status": "Cancelled",
      "id": 13
    }
  ];
}

Play with this code in jsFiddle.


Creates an employee record in a time and attendance database and returns the employee object

It is common practice to use static methods-creators instead of constructors. But this approach is used in cases where you need to convert some objects to the object of desired type. It is the main reason for such methods – conversion from one representation to another.

But you use your CreateNew method for side effects and it is not good in my opinion. If I call CreateNew I'll expect just to get a new instance of the Employee. I don't expect that this method will do something else.

I would create method AddNewEmployeeAndAddToDB but not in the Employee class. Adding to database is not a part of employee functionality.


Things you did well:

Some use of comments.
Use of typedef.

Things you could improve:

Syntax/Styling:

Never use goto.

Yes, there are some rare situations where you may find it necessary to use it.  This is not one of them.

err:
        printf("could not allocate mem\n");
        return 0;

As you can see, all you do in your little block of code here is print out an unspecific error message,  and then exit (using 0, which means the program was successful and is contradictory here).

You should instead abolish the goto's completely.  Print out specific, useful error messages instead, and then return with a unique exit code.

if(mem2 == NULL)
{
    fprintf(stderr, "Error allocating memory to mem2.\n");
    return -2;
}

Your format specifies type int when you are trying to print a string, but the argument has type size_t (basically, unsigned long).  Your format also specifies type unsigned int but the argument has type void *.

printf("\n\t%d\t%d\t\t%x", curr->size, curr->is_available, mem_ptr);

Just change they type you are trying to print out, and cast mem_ptr to an unsigned int.

printf("\n\t%zuu\t%d\t\t%x", curr->size, curr->is_available, (unsigned int) mem_ptr);

You aren't using standard naming conventions with your typedef struct.

typedef struct chunk_header chunk_header;

You should be giving the typedef a specific name, different from the abstract struct name.
You should be capitalizing the name of the type, or adding a _t to the end (though this is reserved in POSIX)

typedef your structs when you declare them.

struct chunk_header {
 struct chunk_header *next; // next pointer on free list
 size_t size;               // the size of this chunk
 bool is_available;         // indicates if this chunk is MEMORY_AVAILABLE or MEMORY_USED
};
typedef struct chunk_header chunk_header;

The typedef means you no longer have to write struct all over the place, as you have used it.  But you can combine these two parts to make things more simple:

typedef struct chunk_header
{
    struct chunk_header *next; // next pointer on free list
    size_t size;               // the size of this chunk
    bool is_available;         // indicates if this chunk is MEMORY_AVAILABLE or MEMORY_USED
} ChunkHeader;

You can simplify your NULL checks.

 if(mem1 == NULL)

Since NULL is defined as (void *)0, we can treat is as a comparison to 0.

if(!mem1)

Parameters/Returns:

You don't accept any parameters for some of your functions, such as main().  If you do not take in any parameters, declare them as void.

int main(void)

You should still return from a function, even if it is declared to return void.

void my_free(void *firstbyte) {
    ...
    return;
}

Compilation:

I can see from some warnings I received that you have not enabled all of your compiler warnings, which you should do.


Most occuring digit in an array of numbers

Outer loop unnecessary
If there are N numbers, each with M digits, your solution is using 10 * N * M operations to compute the solution.  This is because for each digit (e.g. 0, 1, 2), you look at every digit in every number.  But you could make your solution 10 times faster by only looking at each digit once.  All you need to do is to track duplicate digits for each new number.
Here is how that would work:
for (i = 0; i <= 9; i++)
{
    Y[i].number = i;
    Y[i].howmany = 0;
}

for (i = 0; i < sizex; i++)
{
    int foundDigits = 0;

    for (int curNumber = X[i]; curNumber > 0; curNumber /= 10) {
        int digit = curNumber % 10;
        int bit   = 1 << digit;

        if ((foundDigits & bit) == 0) {
            foundDigits |= bit;
            Y[digit].howmany++;
        }
    }
}

Here, foundDigits is a bitmask which contains a 1 bit for each digit already seen in the current number.  This is what is used to track duplicate digits.  When a non-duplicate digit is seen, we can just increment the Y[digit].howmany array directly.
struct numbers unnecessary
Actually, the numbers structure is unnecessary because Y[i].number is always equal to i.  So you can just have Y be an array of integers holding the count of each digit.  Wherever you currently use Y[i].number, just replace that with i instead.


Create a probability list which sums up to one

Binary floating point numbers cannot represent most numbers exactly,
a small error cannot be avoided when computing the sum of probabilities:

>>> probs = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
>>> sum(probs)
0.9999999999999999

For more information, see

Is floating point math broken? on Stack Overflow,
What Every Computer Scientist Should Know About Floating-Point Arithmetic.

Actually one can see in the source code of mtrand.pyx that this rounding error is cared for in the choice() function:

        atol = np.sqrt(np.finfo(np.float64).eps)
        # ...
        p = <ndarray>PyArray_ContiguousFromObject(p, NPY_DOUBLE, 1, 1)
        pix = <double*>PyArray_DATA(p)
        # ...
        if abs(kahan_sum(pix, d) - 1.) > atol:
            raise ValueError("probabilities do not sum to 1")

An error is only raised if the sum differs from 1.0 by more than
some “machine epsilon.”

We also  see  that NumPy uses the
Kahan summation
to add the probabilities, an algorithm which significantly reduces the
numerical error. I don't know if NumPy publicly exposes the Kahan summation
(I could not find it). But it is not difficult to implement, here is a
version from Rosetta Code:

def kahansum(input):
    summ = c = 0
    for num in input:
        y = num - c
        t = summ + y
        c = (t - summ) - y
        summ = t
    return summ

which would then be used as

totalScore = kahansum(item['strength'] for item in data)

Example:

>>> probs = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
>>> sum(probs)
0.9999999999999999
>>> kahansum(probs)
1.0

The computation of the probResult array can be simplified. First note that

value = (100.0/totalScore)*item['strength']
value2 = (1.0/100)*value

is equivalent to

value2 = item['strength'] / totalScore

and that would work for the case item['strength'] == 0 as well:

for item in data:
    value2 = item['strength'] / totalScore
    probResult.append(value2)

And now you can replace the loop with a list comprehension:

def makeProbabilities(data) -> list:
    totalScore = kahansum(item['strength'] for item in data)
    return [ item['strength'] / totalScore for item in data ]

Finally note that according to the Python naming conventions, function
and variable names should be “snake case”: make_probabilities, total_score.


The operation count can be reduced, by using that some bits are moved by the same distance. Bits 0, 1 and 2 can be shifted left in one go. 5 and 6 can be shifted right in one go.

Some shift/bitwise-OR combinations can be written as lea, that would be bad for Pentium 4 but very good on Ryzen and Ice Lake and fine on Haswell/Skylake. Since this is 64bit code it is best to use lea with a 64bit address and then implicitly discard the top half of the result by writing it to a 32bit destination, explicitly using a 32bit address would generate a useless address size override prefix.

The 8 bit operations have a disadvantage on several CPUs that limits their efficiency, they're usually not inherently slow per-se but the problem is that writing to the "low" 8 bit register may have a dependency on the whole register (the "high" registers are an other story and have their own issues), something like mov al, bl doesn't "decouple" the new al from the old al on such CPUs because the write is merged into rax. So unless there is a good reason for writing to an 8 bit register, I would recommend avoiding it, to avoid such odd edge cases of the microarchitecture.

Here is a possible implementation:

    ; keep bit 7 at bit 7
    mov     eax, ebx
    and     eax, 128
    ; shift bits 0, 1, 2 left by 1 and combine
    mov     ecx, ebx
    and     ecx, 7
    lea     eax, [rax + 2*rcx]
    ; shift bit 4 left by 2 and combine
    mov     ecx, ebx
    and     ecx, 0x10
    lea     eax, [rax + 4*rcx]
    ; shift bit 3 right by 3 and combine
    mov     ecx, ebx
    and     ecx, 8
    shr     ecx, 3
    or      eax, ecx
    ; shift bits 5 and 6 right by 1 and combine
    shr     ebx
    and     ebx, 0x30
    or      eax, ebx
    ; store result
    mov [A], al


A nicer way to do the same would be to use a dictionary:

def hello(hvd, coord):
    d = {'h': ["789", "456", "123"],
         'v': ["741", "852", "963"],
         'd': ["159", "753"]}
    try:
        print d[hvd][coord-1]
    except IndexError:
        print "coord is wrong"
    except KeyError:
        print "hvd is wrong"


Improved manipulation of a 2D array for a game map

I like your question, games are one the best ways to get to know a language better.

From a once over:

Unless you have other 2d searching needs elsewhere, I would have called indexOf2d as findRoomLocation
I would have made findRoomLocation a part of worldMap
You are both returning the found location with index and setting the globals outputA  and outputB. Setting those globals is considered very bad practice.
If you can exit early from loops (you could exit once you found a room), then I would advise against use some and go for for instead
Also, indexOf is slower than rolling your own search so I would use 2 old school loops

worldMap.findRoom2 = function( room ){
  var notFound = [-1, -1],
  row;

  if (!Array.isArray(this)) {
    return notFound;
  }

  for( var x = 0 , xLength = this.length ; x < xLength ; x++ ) {
    row = this[x];
    if (!Array.isArray(row)) {
      continue;
    }
    for( var y = 0 , yLength = row.length ; y < yLength ; y++ ) {
      if( row[y] == room ){
        return [x,y];
      }
    }
  }
  return notFound;
};

If this was for a personal non production project I would settle for something like this

worldMap.findRoom = function( room )
{
  var x = -1, y;
  while( this[y=-1,++x] )
    while( this[x][++y] )
      if( this[x][y] == room )
        return [x,y];
  return [-1,-1]; //Not found
};

All in all, I would counter propose something like this:

// noprotect
var room1 = [
    [1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1],
    [1, 1, 1, 1, 1, 1, 1]
];

var room2 = [
    [2, 2, 2, 2, 2, 2, 2],
    [2, 2, 2, 2, 2, 2, 2],
    [2, 2, 2, 2, 2, 2, 2]
];

var room3 = [
    [3, 3, 3, 3, 3, 3, 3],
    [3, 3, 3, 3, 3, 3, 3],
    [3, 3, 3, 3, 3, 3, 3]
];

var room4 = [
    [4, 4, 4, 4, 4, 4, 4],
    [4, 4, 4, 4, 4, 4, 4],
    [4, 4, 4, 4, 4, 4, 4]
];

var worldMap = [
    [room1, room2],
    [room3, room4]
];

var outputA;
var outputB;
var currentMap = room2;

worldMap.findRoom = function( room ){
  var notFound = [-1, -1],
      row;

  if (!Array.isArray(this)) {
        return notFound;
  }

  for( var x = 0 , xLength = this.length ; x < xLength ; x++ ) {
    row = this[x];
    if (!Array.isArray(row)) {
        continue;
    }
    for( var y = 0 , yLength = row.length ; y < yLength ; y++ ) {
      if( row[y] == room ){
        return [x,y];
      }
    }
  }
  return notFound;
};

worldMap.findRoomAlt = function( room )
{
  var x = -1, y = 0;
  while( this[y=-1,++x] )
    while( this[x][++y] )
      if( this[x][y] == room )
        return [x,y];
  return [-1,-1]; //Not found
};

worldMap.findNextRoom = function( directionName ){
  //dx & dy -> Delta on x and y axis
  var directions = {
    down : {  dx : 1 , dy : 0 }
  };
  var direction = directions[directionName];
  //Paranoia
  if(!direction){
    //Do something, I would actually throw an error
  }

  var location = this.findRoom(currentMap);
  location[0] += direction.dx;
  location[1] += direction.dy;
  return this[location[0]][location[1]];
};

// show the existing currentMap
console.log('existing currentMap = ' + currentMap);

// get the index of the currentMap within the worldMap
console.log(worldMap.findRoom(currentMap));

// use the index value outputs to calculate the next room 'down' the worldMap array

console.log('next room down: ' + worldMap.findNextRoom('down'));

currentMap = worldMap.findNextRoom('down');
console.log('new currentMap = ' + currentMap);


Interpret a date from a string of digits

Exception

If your algorithm cannot find a date, it is easier to raise an Exception than to return ''. Returning sentinel values instead of exceptions can lead to unexpected behaviour if the user of this function does not test for this sentinel value.

comments

Comments should explain why you did something, not how. # Take the last four digits tells you nothing more than the code itself. I would rather comment at field[-4 - i:n - i] why you did n - i instead of just -i.

nesting

Instead of nesting a number of if-clauses, it can be better to test the negative of the condition, and continue, so the rest of the code is less nested.

match

Don't test condition is True. Just do condition. In Python a lot of values can act as True or False in tests.

Your match is never used anyway; the moment you set it to True, you also return the result, so a while True: would have sufficed here.

field

This is a very unclear variable name. This method excepts a date in string format, so why not call the argument like that?

Return type

Your code does 2 things now. It looks for a date in the string, and converts that date to another format. It would be better to separate those 2 things, and return a datetime.datetime instead, and let the caller of this method worry about formatting that correctly.

while True

You use a while True-loop, with an incrementing counter. A better way to do this would be to either use for i in range(...) or using itertools.count: for i in itertools.count(). In this case you know there will be no more than len(field) - 7 iterations, so you might as well use that.

Revert the algorithm

You explicitly test whether the substring is 8 characters long, and then if it is in the right format. By changing the while True to the for-loop, you know the substring will be 8 characters long. Then it makes sense to first try to convert it to a datetime, and then check whether the year is correct:

def format_dates2(date_string):
    n = len(date_string)
    for i in range(n - 7):
        sub_string = date_string[-(8 + i) : n - i]
        # not just -i because that fails at i==0
        try:
            date = dt.strptime(sub_string, "%d%m%Y")
        except ValueError:
            continue
        if not (1919 <= date.year <= 2019):
            continue
        return date
    raise ValueError("Date not in the correct format")


Improve efficiency for finding word frequency in text

You're right, the basic point to make this program faster is to store the line numbers in the first pass.

The for word in list loop will run many times. The count depends on how many words you have. Every time it runs, it reopens and rereads the whole input file. This is awfully slow.

If you start using a more generic data structure than your Counter(), then you'll be able to store the line numbers. For example you could use a dictionary. The key of the dict is the word, the value is a list of line numbers. The hit count is stored indirectly as the length of the list of line numbers.

In one pass over the input you can populate this data structure. In a second step, you can properly sort this data structure and iterate over the elements.

An example implementation:

from collections import defaultdict
import sys

if __name__ == "__main__":
    hit = defaultdict(list)
    for lineno, line in enumerate(sys.stdin, start=1):
        for word in line.split():
            hit[word.lower()].append(lineno)
    for word, places in sorted(hit.iteritems(), key=lambda (w, p): (-len(p), w)):
        print len(places), word, ",".join(map(str, sorted(set(places))))

A few more notes:

If you open() a file, close() it too. As soon as you're done with using it. (Or use the with statement to ensure that it gets closed.)
Don't hardcode input/output file names. Read from standard input (sys.stdin), write to standard output (sys.stdout or simply print), and let the user use his shell's redirections: program <input.txt >output.txt
Don't litter standard output with progress messages. If you absolutely need them use sys.stderr.write or get acquainted with the logging module.
Don't use regular expressions, unless it's necessary. Prefer string methods for their simplicity and speed.
Name your variables in lower_case_with_underscores as PEP8 recommends.


Typo

This probably happened when you cut and pasted your code, but your contains function had a few lines switched in their ordering.

Operations taking O(size) time, could be faster

Your add, remove, and contains operations all do approximately the same thing.  They create a hash where the hash contains up to bpe bits set in an array of size bits.  Then they iterate over the size bits.

The important thing here is that bpe is much smaller than size.  For any size larger than 64, bpe will be sqrt(size).  Since only bpe bits are set, you only need to take O(bpe) time instead of O(size) time.

Operations using 32x the required space

Also, when you create your array of bits, you use one int per bit even though you only ever set the int to 1 or 0.  If size is very large, such as 100 million, you could be using 400 MB when you only need to be using 12.5 MB.

Putting it all together

I rewrote your functions, and also combined all of their functionality into the hashing function since all 3 functions were so similar.  In the new code, the loops only run through bpe iterations instead of size iterations.  I also packed the bits to minimize the space used.

/**
 * Performs one of three operations: add, remove, or contains.
 *
 * @param        v                The element to add/remove/test.
 * @param        operation        1 = add, -1 = remove, 0 = contains
 * @return                        If operation is "contains", returns true
 *                                if v is contained in the bloom filter, and
 *                                false if it is not.  If operation is
 *                                anything else, returns true.
 */
private boolean hashOperation(V v, int operation)
{
    Random r         = new Random(v.hashCode());
    int[]  bitsFound = new int[(size+31)/32];

    for(int i=0;i<bpe;i++){
        int bitNum   = r.nextInt(size);
        int bitIndex = bitNum >> 5;
        int bit      = (1 << (bitNum & 31));
        if ((bitsFound[bitIndex] & bit) == 0) {
            // New bit.  Perform operation.
            bitsFound[bitIndex] |= bit;
            if (operation == 0) {
                if(data[bitNum] == 0)
                    return false;
            } else {
                data[bitNum] += operation;
            }
        }
    }
    return true;
}

public boolean remove(V v)
{
    if (contains(v)) {
        hashOperation(v, -1);
        count--;
        return true;
    }
    return false;
}

public void insert(V v)
{
    hashOperation(v, 1);
    count++;
}

public boolean contains(V v)
{
    return hashOperation(v, 0);
}

Addendum: Testing the fastest way to hash using maaartinus's method

Maaartinus showed a fast way to hash using just 2 hash functions.  I decided to test out 4 variants:

Using a power of 2 size (should be fastest)
Using modulus operator every loop (should be slowest)
Using a compare + subtract
Doing the subtract without using any compare

Note, the following is all done in C:

Variant 1

All variants use the following setup.  The thing that varies will be in the loop:

uint32_t hash, hash1, hash2, size;

size  = rand();
hash1 = rand() % size;
hash2 = rand() % size;

hash = hash1;
uint32_t sizeMask = 0x3fffffff;  // Variant 1 only
for (i=0;i<iterations;i++) {
    hash = (hash + hash2) & (sizeMask);
}

Variant 2

for (i=0;i<iterations;i++) {
    hash = (hash + hash2) % size;
}

Variant 3

// If size is < 0x80000000
for (i=0;i<iterations;i++) {
    hash += hash2;
    if (hash > size)
        hash -= size;
}

// This works for all cases (and has the same runtime as the one above)
for (i=0;i<iterations;i++) {
    if (size - hash > hash2)
        hash += hash2;
    else
        hash -= (size - hash2);
}

Variant 4

// In C, >> is not guaranteed to do an arithmetic shift, so
// we use a logical shift instead.  The compiler optimizes this code
// to do an arithmetic shift anyways.  In java, change the ">>" here
// to a ">>>".
for (i=0;i<iterations;i++) {
    hash += hash2;
    hash -= (-((size - hash) >> 31)) & size;
}

// In java, where >> is arithmetic shift, you could write it like this.
for (i=0;i<iterations;i++) {
    hash += hash2;
    hash -= ((size - hash) >> 31) & size;
}

Timing results

Running each variant for 1 billion iterations (results in seconds):

Variant 1 (& power of 2): 0.58
Variant 2 (% size)      : 6.56
Variant 3 (if, subtract): 1.25
Variant 4 (no if)       : 1.44

However, this is a bit misleading because there was nothing in the loop except for the hash calculation.  If I add in the part where we use the hash on each loop:

    bits[hash>>5] |= (1 << (hash & 31));

Then the timings all become much closer:

Size = 1 billion
Variant 1 (& power of 2): 15.92
Variant 2 (% size)      : 18.92
Variant 3 (if, subtract): 17.48
Variant 4 (no if)       : 16.76

Size = 4 million
Variant 1 (& power of 2): 2.48
Variant 2 (% size)      : 7.11
Variant 3 (if, subtract): 2.58
Variant 4 (no if)       : 2.52

Size = 256K
Variant 1 (& power of 2): 1.24
Variant 2 (% size)      : 6.64
Variant 3 (if, subtract): 1.79
Variant 4 (no if)       : 1.95

So it appears that at large sizes, the memory manipulation of the bit array will be the limiting factor and not the hash calculation.  So you can choose from the above 4 variants based on:

Whether you can force the size to be a power of 2 or not.
How much you need that extra bit of speed versus how clear/simple you want your code to be.


Not a Go expert but I think you should be able to define your array of values as you declare it :

triple := [...]int{m*m - n*n, 2 * m * n, m*m + n*n}

Source


The optimal solution to this is significantly simpler than what you have. We'll get to that in a second. First, let's examine your code:

Cop should not begin with a capital letter. Variables should be snake_case. Furthermore, cop is a bad name. At the least it should be copy (don't shorten names, it only makes things less clear--unless the shortening is a broadly understood acronym like HTML). But copy of what? From that name, I can't tell. I have to go hunting for the initialization of it (and all of the mutations to it) to figure out what it actually does. What are J, Q, and W? Again, they should be lowercase. But also, letters make awful variable names (except for maybe i and j for numeric loop variables).
n
Your code either has or is precariously close to an off by one issue with all of the +1ing that you're doing. Why is it print(count + 1) and not print(count)?
count = count + len(K) should be count += len(K)
[[1 for i in range(num)]] can just be [[1] * num]
PEP8 your code. It is messy and has little whitespace, which makes it hard to follow.
con is a poorly named function. What does that even mean? Clearly it doesn't count anything because you have to use it in a loop to get the answer to the problem.
Why are you maintaining Q as a list to only use it in T = list(set(Q))? Q should just be a set from the start, that's more efficient.
There's no need to do the list in T = list(set(Q)). You can iterate over T = set(Q) (without the need to create a list).
You need to comment this code. The fact that you need to spend a paragraph explaining how your code works is a pretty good sign of this. It would take a long time for me to walk through this an understand how each piece works. Comment things that aren't clear. Comment why not how. Add docstrings to functions. I wouldn't even know how to just use your code to get the answer to the problem. I assume setting num, but it's unclear to me if that gives the answer for 5 or 4, given that you do range(num) which is exclusive not inclusive (it's [0, 1, 2, 3, 4])

That's a not exhaustive list of issues with your code as it exists. But, we need to start from scratch, because what you have is way too complicated for the task at hand. If we take a step back and rethink the problem, we can come up with a much more elegant solution.

Let's start by looking at the problem:

  It is possible to write five as a sum in exactly six different ways:

4 + 1
3 + 2
3 + 1 + 1
2 + 2 + 1
2 + 1 + 1 + 1
1 + 1 + 1 + 1 + 1

  How many different ways can one hundred be written as a sum of at least two positive integers?

Here's a question: can we write out all of the ways to write four as a sum? Sure, it looks like:

3 + 1
2 + 2
2 + 1 + 1
1 + 1 + 1 + 1

Hmm, that looks suspiciously similar to the list above. Let's rewrite the sums for five. I'll put the parts that are unique to five in parenthesis:

(4 + 1)
(3 + 2)
3 + 1 (+ 1)
2 + 2 (+ 1)
2 + 1 + 1 (+ 1)
1 + 1 + 1 + 1 (+ 1)

We should notice a pattern here. Namely, the sums for five are the sums for four with + 1 added and an additional 2 new ways. The 2 new ways are just combinations of the first sum of four + 1. 3 + 1 + 1 if you merge the 1 + 1 is 3 + 2 and 3 + 1 + 1 if your merge the 3 + 1 is 4 + 1.

Similarly, we can write the sums for four as the sums of three with some extras added on:

(3 + 1)
(2 + 2)
2 + 1 (+ 1)
1 + 1 + 1 (+ 1)

This pattern continues all the way down to one where there is only one sum: 1 itself.

Hopefully, this pattern is hinting to you that there is a recurrence relation we could take advantage of. Namely, if we know the number of sums for 5 is the number of sums for 4 plus some extras. This goes all the way down to 1. This should hint to you that a recursive solution may be appropriate. However, there's a catch. While we can implement it recursively, it turns out that the dependency chain isn't linear. Each answer depends on every answer before it. Because of this, even though you could compute the answer recursively, it would be inefficient, because it would require recomputing the answer for every number below. But we're really close.

As an aside, there are other (perhaps more natural) phrasings of this problem, which may make this +1 structure more obvious. Consider you have n pennies. How many unique groups of all n pennies can you make (each group is like one of the terms in the addition). For example, if n = 5 then a group of 4 pennies and a group of 1 pennies (one of the sums for 5) is like 4 + 1.

The efficient solution recalls that we can take a dynamic programming approach where we memoize answers to smaller problems. We can do so by accumulating a list of the previous answers num_ways_to_sum and build this up from 1 up to the desired number:

def count_number_of_ways_to_sum(target_sum):
    """
    Compute the number of unique ways (order does not matter) of summing to target_sum
    using numbers from 1 to target_sum, exclusive.
    """
    if target_sum <= 1:
        raise ValueError('there are no numbers greater than 0 less than the target sum')

    # ith index contains the answer to the (i + 1)th subproblem
    num_ways_to_sum = [0] * (target_sum + 1)

    # There is only 1 way to sum to 1
    num_ways_to_sum[0] = 1

    # Consider sums involving numbers in [1, target_sum)
    for i in range(1, target_sum):
        # Every subproblem j >= i depends on i
        for j in range(i, target_sum + 1):
            # The number of ways to sum j includes all the ways to sum i
            # with all the ways to sum (j - i) appended (like the parenthesis)
            num_ways_to_sum[j] += num_ways_to_sum[j - i]

    return num_ways_to_sum[target_sum]

Note a few things:

We document and properly name the function
We are careful with domain. We raise ValueError when we are given an impossible target_sum. (1 has no solution, because there are no counting numbers less than 1 but greater than 0)
We name variables appropriately. It is clear what num_ways_to_sum is. We may even consider renaming i and j, although the comments perhaps make them clear enough (and given that they are only used over 3 lines, this information is close together and intelligible as a single unit.
We comment!
We use python standard library features appropriately (like lists, range, etc.)

Now, you should definitely test this with the unittest package.


g_server.receiveData();

Does this actually handle all of the message parsing?

As in: Is it really necessary to handle the full netcode in the main thread?

At least unpacking and sorting all inputs to a thread safe queues (e.g. lockfree queue from boost) with unresolved, symbolic references should be easily possible inside the worker threads handling the connections.

Validation of conflicting inputs, or orphaned references can't be avoided in the main loop. But parsing the packages and preliminary filtering can.

releasePointersCounter += delay;
if (releasePointersCounter >= 1000) {
    // This function simply calls a "delete" to removed objects from the game
    releaseObjects();
}

Not using C++11 shared pointers with builtin reference counting? Respectively, any specific reason (like cyclic references) which would prevent these from working?

// Sends all pending packets to be sent to active connections
g_server.sendAllData();

I must assume the world state, respectively the list of updates is not buffered, so this reads from the live game state?

Writing out all changes to the game state to a transaction log (with all dangling pointers resolved, implemented as a list with a single producer and multiple non-consuming iterators), and then walking the transaction log for each active connection asynchronously removes this load from the main loop as well.

There is no good reason why this should be allowed to stall the main loop.

int64_t connectionsTimeCounter = 0;
int64_t decayTimeCounter = 0;
int64_t releasePointersCounter = 0;
int64_t creaturesTimeCounter = 0;

Why are these variables write-only, apart from the initial check when to run the corresponding logic first? Respectively why does the main loop wait a full second before starting any of the workers at all?


You could improve your first example by returning early and moving variable definitions nearer to first use. I prefer this to the use of gotos but maybe I've just been conditioned against their use for too long:

BOOL read_file(PCHAR filename, PBYTE *outdata, DWORD *outsize)
{
    HANDLE h = CreateFileA(filename, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if(h == INVALID_HANDLE_VALUE) {
        return FALSE;
    }
    BOOL success = FALSE;
    if((*outsize = GetFileSize(h, NULL)) != INVALID_FILE_SIZE)
    {
        if((*outdata = malloc(*outsize)) != NULL)
        {
            DWORD check = 0;
            if(ReadFile(h, *outdata, *outsize, &check, NULL) && check == *outsize)
            {
                success = TRUE;
            }
            else { free(*outdata); }
        }
    }
    CloseHandle(h);
    return success;
}

Another approach is to split into two functions, thereby removing the need for nesting and gotos.  The compiler can optimize away the function call (eg by inlining) if read_file_data is made static, but this inner function might possibly be useful in itself:

static BOOL read_file_data(HANDLE h, DWORD size, PBYTE *outdata)
{
    *outdata = malloc(size);
    if (*outdata == NULL) {
        return FALSE;
    }
    DWORD size_read = 0;
    if (ReadFile(h, *outdata, size, &size_read, NULL) && (size_read == size)) {
        return TRUE;
    }
    free(*outdata);
    return FALSE;
}

BOOL read_file(PCHAR filename, PBYTE *outdata, DWORD *outsize)
{
    HANDLE h = CreateFileA(filename, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
    if (h == INVALID_HANDLE_VALUE) {
        return FALSE;
    }
    BOOL success = FALSE;
    *outsize = GetFileSize(h, NULL);
    if (size != INVALID_FILE_SIZE) {
        success = read_file_data(h, *outsize, outdata);
    }
    CloseHandle(hFile);
    return success;
}

You can also improve the goto method a little by returning early and changing the default success to FALSE (and freeing the buffer on error).

BOOL read_file2(PCHAR filename, PBYTE *outdata, DWORD *outsize)
{
    BOOL success = FALSE;
    HANDLE hFile = CreateFileA(filename, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);

    if(hFile == INVALID_HANDLE_VALUE) {
        return FALSE;
    }
    if((*outsize = GetFileSize(hFile, NULL)) == INVALID_FILE_SIZE) {
        goto _end;
    }
    if((*outdata = malloc(*outsize)) == NULL) {
        goto _end;
    }
    DWORD check = 0;
    if(!ReadFile(hFile, *outdata, *outsize, &check, NULL) || (check != *outsize)) {
        free(*outdata);
    }
    else success = TRUE;

_end:
    CloseHandle(hFile);
    return success;
}


The changes I will propose are from ruby-style-guide since that is accepted by the majority of Ruby developers.

What you can improve:

Remove initialize method since there is no need for it in your case.
Use white space to separate groups of logic, something along these lines:

raise 'only accepts a string' unless string_of_numbers.is_a?(String)

string_array = string_of_numbers.split(/[^0-9-]+/)   integer_array = string_array.map(&:to_i)

raise "cannot accept negatives -#{check_for_negatives(integer_array)}" if  check_for_negatives(integer_array)

integer_array.inject(0){|sum,x| x <= 1000? sum + x : sum }

It's not a good practice to hard code the data in the name, instead of string_of_numbers replace with numbers, and instead of integer_array you can figure something else (naming is hard).
Use public and private API methods, since int_add is the only method your class is using, you can rename it to add, and make check_for_negatives private so the reader will know to rely only on add as a stable method.
Ruby returns the last expression, so you don't need an assignment for return negatives_string = negatives_array.join(","), it can be negatives_array.join(",")
You can reduce the check_for_negatives method to this:

def check_for_negatives(numbers)
  negatives = numbers.select{ |i| i < 0 }
  # Ruby return the last statement which is the negatives string,
  # or nil, that is treated as false
  negatives.join(",") if negatives.length > 0
end


Maximum product of 3 integers in an int array using Python - follow up

This version feels much more readable, especialy since you dropped half of your comparisons.

However, I really liked @JoeWallis use of heapq. I would however use heappushpop which will provide all the comparisons you are doing at once.

But you will need to manually extract out the maximum value of max_seen since you have no guarantee anymore on its position.

You can thus write:

from heapq import heappushpop

def highest_product(list_of_ints):
    max_seen = [float("-inf")] * 3
    min_seen = [float("-inf")] * 2

    for x in list_of_ints:
        heappushpop(max_seen, x)
        heappushpop(min_seen, -x)

    # No change of sign since we changed it twice (once for each element)
    max_product_candidate_one = min_seen[0] * min_seen[1] * max(max_seen)
    max_product_candidate_two = max_seen[0] * max_seen[1] * max_seen[2]

    return max(max_product_candidate_one, max_product_candidate_two)


Project Euler 378: Count ordered triplets whose triangular numbers have decreasing number of factors

I am new to python; I only know at most the loops,list comprehension,
  functions, and some class. I am sure this problem can be solved more
  simply.

  ...

  Is there an improvement I can make with my current knowledge? If not,
  what should I be learning about next?

Loops and arrays are all you really need to solve almost any Project Euler challenge. The only exceptions are the ones which also require reading an input file. The point of PE is not to push your knowledge of language features, but to push your mathematical skills and knowledge of data structures and algorithms.

I would advise you to solve at least 40 of the first 50 challenges before you start looking at challenges beyond 100, because the early challenges introduce key techniques which you will use many times in later challenges. Trying to solve #378 before #12 is simply doing things in the wrong order.

The most important of the key techniques is dynamic programming. So many PE problems can only be solved in a reasonable time by exploiting shared structure. In this challenge, for example, you use the "number of divisors" function \$d(n)\$. The structure of this function can be summed up as $$\begin{align}
d(p^a) &= a+1&\qquad \textrm{when }p\textrm{ is prime} \\
d(mn) &=d(m)d(n)&\qquad \textrm{when }\gcd(m,n)=1
\end{align}$$
Exploiting that structure by adapting the sieve of Eratosphenes will allow you to calculate an array containing dt(n) for every \$1 \le n \le 60000000\$ in less than a minute.

Similarly, exploiting the structure of \$1 ≤ i < j < k ≤ n\$ and \$\mathit{dT}(i) > \mathit{dT}(j) > \mathit{dT}(k)\$ will allow you to reduce the current \$O(n^3)\$ runtime of that accumulation phase to \$O(nw)\$ where \$w = \max \{dT(i) : 1 \le i \le n\}\$. That's enough to solve the problem in about 40 minutes, using only basic data structures.

To solve the problem in under two minutes you will need an advanced data structure, but that's more than enough spoiler. Solve the problem at all and you'll get access to the forum thread where people explain their solutions.


The snippet e.each{|k,v| m[k] = v} does the same thing as the standard library method Hash#merge!, so the shortest way to write your solution is probably:

animals.reduce({}, :merge!)

Note that with this solution, if two hashes in the source array have duplicate keys, those coming later in the array will take priority and overwrite the duplicate keys from earlier in the array:

[{"egrets" => 17}, {"egrets" => 21}].reduce({}, :merge!)  # => {"egrets"=>21}

Also be aware that merge! is destructive to the original hash, which is fine here since we don't reuse the literal input to reduce. There is a non-destructive version, merge, which is better when the input needs to be preserved.


Simple for loop to calculate headers size based on a specific ratio

This is what your code produces and I'm wondering if this really what you want:

"
h1 { font-size: 1.0957612059345655e+45px; }
h2 { font-size: 2.602367950332107e+22px; }
h3 { font-size: 126822144384px; }
h4 { font-size: 279968px; }
h5 { font-size: 416px; }
h6 { font-size: 16px; }"

The numbers are huge. Looking at this page, it seems you should not have base inside the multiplication here:

base *= Math.round(base*goldenRatio);

Also, by rounding at this step, you lose precision. It would be better to round right before you put the value in the string. Like this:

var base = 16;
var list = "";
for (var i = 6; i > 0; i--) {
    list = "\nh" + i + " { font-size: " + Math.round(base) + "px; }" + list;
    base *= goldenRatio;
}

I'm also wondering if you really want the \n at the start of the resulting string.
It would seem more natural to put it at the end of each line, like this:

list = "h" + i + " { font-size: " + Math.round(base) + "px; }\n" + list;

Notice also that I added more spaces around the operators in the for.
There is no real standard for this in JavaScript,
but this is a practice I borrow from other languages where it is a standard,
for improved readability. Compare these two versions, I hope you'll agree that this is an improvement:

for(var i=6; i>0; i--){

for (var i = 6; i > 0; i--) {


The function is called BuildIndex, but it could be seen as doing the following:

Measuring its execution time.
Creating an Analyzer.
Creating a Lucene.Net.Store.Directory.
Using the analyzer and the directory to create an IndexWriter.
Iterating files
Loading document body from physical file if it exists at specified location.
Creating a new Document for each file, adding 6 Fields.
Returning execution time.

The first thing that strikes me is that there seems to have a performance concern that shadows the code's intent: is the goal of the method to build an index, or to calculate how long it takes to build an index? The first thing I'd do is return void and remove the first thing in that list - focused code does one thing, it has only a single responsibility in mind.

The next 3 points are closely related and could easily fit inside a method whose job is to return a new IndexWriter:

private IndexWriter CreateWriter(Analyzer analyzer, string path)
{
    var directory = new DirectoryInfo(path);
    var mMapDirectory = new MMapDirectory(directory);

    return new IndexWriter(mMapDirectory, analyzer, IndexWriter.MaxFieldLength.UNLIMITED)
}

The reason this method is taking an Analyzer parameter is because it leaves you the liberty of calling it and passing in a StandardAnalyzer today, and a CustomAnalyzer tomorrow, without changing anything in the method, only the way the analyzer gets instantiated.

In your original code, the version, analyzer path and directory locals all only exist to create the IndexWriter. Extracting the instantiation of this writer into its own method further reduces noise and increases signal. Consider this:

public void BuildIndex(Analyzer analyzer, int pageNum, int pageSize)
{
    using(var writer = CreateWriter(analyzer, @"C:\LuceneIndex"))
    {
        //...
    }
}

Now BuildIndex doesn't care anymore about what Lucene version the analyzer is configured with; not anymore does it care about the actual type of that analyzer either.

The body of the using block declares a bodyBuilder variable that I would have simply called builder. In fact, the StreamReader, is part of implementation details of another method, just like the creation of a new document:

using(var writer = CreateWriter(analyzer, @"C:\LuceneIndex"))
{
    var files = ReaderDAL.FetchBooklistSearchData(pageNum, pageSize);
    foreach (var file in files)
    {
        //Initialize the StringBuilder with 4 MB to accommodate our data:
        var builder = new StringBuilder(4*1024*1024);

        var directory = new DirectoryInfo(file.IndexFilePath);
        if (directory.Exists)
        {
            ReadAllPageFilesInFolder(directory, builder);
        }

        writer.AddDocument(CreateDocument(file, builder.ToString()));
    }

    writer.Optimize();
    writer.Flush(true, true, true);
}

The advantage of a StreamReader is that it allows you to load a large file's content in small chunks.. but using ReadToEnd() is essentially the same as calling File.ReadAllText(), so if the files are easily digestible it would be more easily readable to go that way:

private void ReadAllPageFilesInFolder(DirectoryInfo directory, StringBuilder builder)
{
    foreach (var file in directory.EnumerateFiles())
    {
        // is this file really a PageFile?

        builder.AppendLine(File.ReadAllText(file.FullName));
    }
}

Moving the document creation into its own method makes it easier to later change how a document gets created (new field(s), new settings, etc.):

private Document CreateDocument(WhateverThatTypeIs file, string body)
{
    var result = new Document();

    result.Add(new Field("id", file.BookId.ToString(), Field.Store.YES, Field.Index.NO));
    result.Add(new Field("author", file.Author, Field.Store.YES, Field.Index.ANALYZED));
    result.Add(new Field("annotation", file.Annotation, Field.Store.YES, Field.Index.ANALYZED));
    result.Add(new Field("publisher", file.Publisher, Field.Store.YES, Field.Index.ANALYZED));
    result.Add(new Field("title", file.Title, Field.Store.YES, Field.Index.ANALYZED));
    result.Add(new Field("body", body, Field.Store.YES, Field.Index.ANALYZED));

    return result;
}

That's all nice, but now you want BuildIndex to measure its execution time and return it as a string, right? Wrong.

Let's say the BuildIndex method is located inside an IndexBuilder class; I'd make the method virtual...

public class IndexBuilder
{
    public virtual void BuildIndex(Analyzer analyzer, int pageNum, int pageSize)
    {
        // the above code
    }
}

... and then I'd create a class whose role is to decorate the existing functionality with another functionality, that I can substitute for an IndexBuilder wherever I need one:

public IndexBuilderStopWatchDecorator : IndexBuilder
{
    private readonly IndexBuilder _builder;

    public IndexBuilderStopWatchDecorator(IndexBuilder builder)
    {
        _builder = builder;
    }

    public override void BuildIndex(Analyzer analyzer, int pageNum, int pageSize)
    {
        var watch = StopWatch.StartNew();
        _builder.BuildIndex(analyzer, pageNum, pageSize);
        watch.Stop();
        // log watch.ElapsedMilliseconds,
        // write watch.Elapsed.TotalSeconds to debug output,
        // and/or do whatever the original code is doing with that value.
    }
}


Your code looks OK and will work but I have a few issues with it.

jQuery

You're using $.each in your distinct() function - this is fine, but it means I can now not use your function unless I support jQuery. jQuery is not really known as a general purpose utility library, so if I'm using Angular or Ember, I'm probably not going to want to use your function.

$.each is designed primarily for iterating the DOM rather than general arrays.

Reusability

Your distinct function is not very re-usable. You don't take a function as a parameter for matching, which means I have to rely on your method of determining equality. This might work for your problem space, but it does not for mine - Let us say I want to determine equality between base 64 strings. I cannot use your equality function there, and so if I have an array of base 64 strings I'm buggered.

Comparing JSON strings.

An interesting idea, I will give you that, but this is a bit... weird, and incurs a performance overhead. It certainly threw me off. While there is nothing intrinsically wrong with this approach, it feels dirty.

Behold lodash/uniq!

With all this said and done, there is already a utility function to perform this exact purpose in lodash (arguably the best general-purpose utility library in JavaScript). Granted, it only works on arrays, but so does your example. It also comes with the ability to specify your own matcher, and performs optimizations on sorted arrays.

var testList = [{ foo: 1 }, { foo: 1 }, { bar: 1 }, { bar: 2, baz: 3 }];
console.log(_.uniqBy(testList, function(obj) { return JSON.stringify(obj); });


Grokking the basics of BlockChain with Python

By default mypy doesn't test much. This is because a large part of its philosophy is to allow dynamic and statically typed code at the same time. This is so migrating to mypy is easier and less daunting. Having thousands of errors when you start to port your legacy app is likely to scare a fair few mortals away.

Please use the --strict flag to have typed Python, rather than hybrid Python.

When you use the flag and you type all the functions and methods, you'll notice that there's an issue with Node.prior_node. Currently it's assigned the type Node, but we know that's a lie because we have if prior_node is None.
I personally use --ignore-missing-imports rather than ignoring each import, as they quickly build up over time.
Your __repr__ are non standard.

  Called by the repr() built-in function to compute the “official” string representation of an object. If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment). If this is not possible, a string of the form <...some useful description...> should be returned.

You probably want to be using __str__.
I find it a little confusing that init_vector is being assigned two different things. It would make more sense if you pass in an empty bytes to _ensure_byte_length.

prior_hash = b'' if prior_node is None else prior_node.hash
init_vector = _ensure_byte_length(prior_hash, 16)

You could change the ternary into a getattr.

prior_hash = getattr(prior_hash, 'hash', b'')

I would change Node to a dataclass. Why write code when you can just not?

This would require moving the hash generation into a class method.
I would define an __iter__ method on the node so that we can easily traverse the chain from any node.

This makes the __len__ method of Chain really simple and clean.
I'd rename _ensure_byte_length to _pad. The function has two jobs, pad is well known and allows us to have a much shorter function name.
_ensure_byte_length doesn't need the extra bytes call.
The method Chain.add_node is un-Pythonic. In Python it's standard to return nothing from a function with mutations.

from __future__ import annotations

import dataclasses
from typing import Optional, Iterator

from oscrypto.symmetric import aes_cbc_pkcs7_encrypt as encrypt

@dataclasses.dataclass
class Node:
    prev_node: Optional[Node]
    data: bytes
    hash: bytes

    @classmethod
    def build(cls, key: bytes, data: bytes, prev_node: Optional[Node] = None) -> Node:
        prev_hash = b"" if prev_node is None else prev_node.hash
        hash = encrypt(_pad(key, 32), data, _pad(prev_hash, 16))[1]
        return cls(prev_node, data, hash)

    def __iter__(self) -> Iterator[Node]:
        node: Optional[Node] = self
        while node is not None:
            yield node
            node = node.prev_node

def _pad(bytes_: bytes, length: int) -> bytes:
    return bytes_.ljust(length, b"\x00")[:length]

@dataclasses.dataclass
class Chain:
    node: Node

    def add_node(self, key: bytes, data: bytes) -> None:
        self.node = Node.build(key, data, self.node)

    def __len__(self) -> int:
        return sum(1 for _ in self.node)

def main() -> None:
    chain = Chain(Node.build(b"the key", b"here is a bit of data"))
    chain.add_node(b"P@$$w0rd", b"and here is a bit more data")
    chain.add_node(b"hunter2", b"and finally here is some more")
    print(chain)

if __name__ == "__main__":
    main()


Create a collection of binaries with random set positions

The logic to generate a length between n and 31 has nothing to do with creating a random list of a given length with a given number of set bits. Split them out into multiple functions.

This highlights the usefulness of making the randomness a parameter of the function, so do that too.

Use better names than n and x.

invalidArg is prettier.

Your matchtotalLength` is unneeded; the second path always works and the first isn't common enough to be worthwhile.

A for loop is a much better expression of intent when mutating res, and you can drop a lot of silliness by doing so.

By now we have

let generateBits (rand : Random) (length : int) (set_bits : int) =
    if not (0 <= set_bits && set_bits <= length) then
        invalidArg "set_bits" "must be nonnegative and no biger than length"

    let res = Array.zeroCreate length
    let indices = ResizeArray(seq{0..length - 1})
    for i in 0 .. set_bits - 1 do
        let random_index = rand.Next(0, length - i)
        res.[indices.[random_index]] <- 1
        indices.RemoveAt(random_index)
    res

let createRandomBinary (rand : Random) (max_length : int) (set_bits : int) =
    generateBits rand (rand.Next(set_bits, max_length)) set_bits

and your old function is equivalent to createRandomBinary (Random()) 32.

indices.RemoveAt is slow. Luckily we don't have to do it; we can get a random sample by using an array and just swapping the used elements to the end:

let indices = Array.init length id
for i in 0 .. set_bits - 1 do
    let random_index = rand.Next(0, length - i)
    res.[indices.[random_index]] <- 1

    let item = indices.[random_index]
    indices.[random_index] <- indices.[length - i - 1]
    indices.[length - i - 1] <- random_index
res

Then note that instead you can just set any bits of the array and shuffle it after:

let shuffle (rand : Random) (array : int []) =
    Array.iteri (fun i elem ->
        let index = rand.Next(i, Array.length array)
        array.[i] <- array.[index]
        array.[index] <- elem
    ) array

let generateBits (rand : Random) (length : int) (set_bits : int) =
    if not (0 <= set_bits && set_bits <= length) then
        invalidArg "set_bits" "must be nonnegative and no biger than length"

    let res = Array.init length (fun i -> Convert.ToInt32 (i < set_bits))
    shuffle rand res
    res

let createRandomBinary (rand : Random) (max_length : int) (set_bits : int) =
    generateBits rand (rand.Next(set_bits, max_length)) set_bits


Yet another BrainF*** Interpretter in Haskell

First of all, good job. Now, what can we improve? Let's start with the largest function first, runInstruction:

runInstruction :: Memory -> Instruction -> IO (IO Memory, IO ())
runInstruction mem Nop = return (return mem, return ())
runInstruction (Memory xs (y:[])) Rgt = return (return (Memory (y:xs) [0]), return ())
runInstruction (Memory xs (y:ys)) Rgt = return (return (Memory (y:xs) ys), return ())
runInstruction (Memory (x:xs) ys) Lft = return (return (Memory xs (x:ys)), return ())
runInstruction (Memory [] ys) Lft = return (return (Memory [] (0:ys)), return ())
runInstruction (Memory xs (y:ys)) Inc = return (return (Memory xs ((y+1):ys)), return ())
runInstruction (Memory xs (y:ys)) Dec = return (return (Memory xs ((y-1):ys)), return ())
runInstruction mem@(Memory _ (x:xs)) Out = return (return mem, putChar (chr x))
runInstruction (Memory xs (y:ys)) Inp = return (mem', return ())
                                          where mem' = getChar >>= \z -> return (Memory xs ((ord z):ys))
runInstruction mem@(Memory xs (0:ys)) (Loop _) = return (return mem, return ())
runInstruction mem (Loop prog) = do
                                    (mem1, act') <- runProgram mem prog
                                    mem' <- mem1
                                    (mem'', act'') <- runInstruction mem' (Loop prog)
                                    return (mem'', act' >> act'')

"Memory" handling

Almost all of those functions concern Memory, which isn't obvious. That's because your included all Memory functionality into runInstruction. If you follow that model, you can never use Memory in any other context.

So instead let us write some other functions:

moveRight   :: Memory -> Memory
moveLeft    :: Memory -> Memory
getValue    :: Memory -> Int
setValue    :: Int -> Memory -> Int
modifyValue :: (Int -> Int) -> Memory -> Memory

Instead of data Memory = Memory [Int] [Int], we're going to use

data Memory = Memory [Int] Int [Int]

That way we will always have a current value. The initial empty memory is the same as yours, except for the pivot value

emptyMemory = Memory [] 0 []

and moving left or right stays mostly the same:

moveRight (Memory []     v rs) = Memory [] 0 (v:rs)
moveRight (Memory (l:ls) v rs) = Memory ls l (v:rs)
moveLeft (Memory ls v [])     = Memory (v:ls) 0 []
moveLeft (Memory ls v (r:rs)) = Memory (v:ls) r rs

getValue (Memory _ v _ )       = v
setValue v                     = modifyValue (const v)
modifyValue f (Memory ls v rs) = Memory ls (f v) rs

What's nice about those functions is that we can easily check them with QuickCheck or other testing frameworks, since they are all pure. For example, we can check that

prop_set_get x mem = getValue (setValue x mem)   == x
prop_set_set x mem = setValue (setValue x mem)   == (setValue x mem)
prop_get_set   mem = setValue (getValue mem) mem == mem

This is hard to check with your current variant.

By the way your variant was completely opaque whether the left or the right list would provide the pivot element. Our new Memory makes sure that we cannot do it wrong, and that v in Memory _ v _ always exists.

How does runInstruction look with those changes?

runInstruction :: Memory -> Instruction -> IO (IO Memory, IO ())
runInstruction mem Nop = return (return mem, return ())
runInstruction mem Rgt = return (return (moveRight mem), return ())
runInstruction mem Lft = return (return (moveLeft  mem), return ())
runInstruction mem Inc = return (return (modifyValue succ mem), return ())
runInstruction mem Dec = return (return (modifyValue pred mem), return ())
runInstruction mem Out = return (return mem, putChar (chr (getValue mem)))
runInstruction mem Inp = return (mem'      , return ())
                           where mem' = getChar >>= \z -> return (setValue (const z) mem)
runInstruction mem (Loop prog)
  | getValue mem == 0 = return (return mem, return ())
  | otherwise         = do
                        (mem1, act') <- runProgram mem prog
                        mem' <- mem1
                        (mem'', act'') <- runInstruction mem' (Loop prog)
                        return (mem'', act' >> act'')

Almost readable, right? But something is still off.

Better types

Your result type of runInstruction doesn't really capture what you want to do. You want to run an instruction and get a new Memory, maybe with some output on your console, or some user input taken. But even without that information, we'll notice that you never user the IO () part of your result, except for putChar. That line could also get written as

runInstruction mem Out = putChar (getValue mem) >> return (return mem, return ())

Therefore we don't need it at all. So we could use IO (IO Memory) instead (spoiler: don't):

runInstruction :: Memory -> Instruction -> IO (IO Memory)
runInstruction mem Nop = return (return mem)
runInstruction mem Rgt = return (return (moveRight mem))
runInstruction mem Lft = return (return (moveLeft  mem))
runInstruction mem Inc = return (return (modifyValue succ mem))
... don't use this!

But we're stopping short before the finish line. As I said, you just want to change the memory in your program, and maybe, maybe want to show or take some text. So runInstruction's correct type would be:

runInstruction :: Instruction -> Memory -> IO Memory

It's implementation stays the same, except that we lose several returns:

runInstruction :: Instruction -> Memory -> IO Memory
runInstruction i mem@(Memory _ v _) = case i of
  Lft       -> return (moveLeft mem)
  Rgt       -> return (moveRight mem)
  Inc       -> return (modifyValue succ mem)
  Dec       -> return (modifyValue pred mem)
  Out       -> putChar v >> return mem
  Inp       -> (\z -> setValue (ord z) mem) `fmap` getChar
  Loop _ | v == 0 -> return mem
  Loop p    -> runProgram p mem >>= runInstruction i
  _         -> return mem

I've switched the order of Memory and Instruction so that both occurences of Memory are closer to each other. Of course, this means that we need to change runProgram too, but that change isn't really large:

runProgram :: Program -> Memory -> IO Memory
runProgram []     mem = return mem
runProgram (i:is) mem = runInstruction i mem >>= runProgram is

I think the last lines in both functions showed why I moved Memory to be the last argument.

You're off course free to change runInstruction back to your pattern matching variant.

Time for some exercises

Now that we have something that's easier to read, it would be time to refactor. So here are some exercises:

Brainfuck only supports integers from 0-255 officially, and it wraps. This should get reflected in our functions. With our new program structure, there is only one single point where you need to change something. Where? And why?
Instead of Nop, use Comment Char. That way, you can still have a look at the code.
Try to make a variant of runProgram that takes a String to simulate user input and produces a String to simulate output.
Try to parse more variants of Brainfuck, for example charmander, whitespace or Ook?!.