Programming Concepts

Computer programs

Application Purpose

Applications are built using one or more programming languages and are typically designed to solve a specific problem. They often include multiple features that allow users to interact with the system, sending or receiving information through input/output devices such as a console display or a printer.

Application Execution

The execution of a program aims to resolve a problem and communicate the result, or it can produce a physical effect, such as displaying a message on a monitor or controlling a robotic arm. A program can be designed to be executed a single time or multiple times as needed.

Code base

A code base is all the source code that is used to build a particular software system or application. It can consist of one or more projects. The term has been around for many years and is a valuable metric for understanding the complexity of a system.

For the first part of your career, you will probably work in relatively small code bases (less than 100,000 lines of code), and may never encounter a large code base. However, it's not uncommon for large systems like Windows, Linux, or major web services to have code bases exceeding several million lines of code.

Example: Let's say you are analyzing a code base with 500,000 lines of code. This is considered a medium-sized code base, and working with it requires specialized tools and processes.

Programming symbols

Letters and Digits

Letters (a-z, A-Z) and digits (0-9) are used to create identifiers, which are the names we give to variables, functions, and other entities in our programs. In most languages, identifiers must start with a letter or underscore.

Operators

Operators are symbols that perform specific operations on variables and values. Common operators include:

• Arithmetic: + (addition), - (subtraction), * (multiplication), / (division), % (modulo)
• Assignment: = (assign a value to a variable)
• Comparison: == (equal), != (not equal), < (less than), > (greater than)
• Logical: && (AND), || (OR), ! (NOT)

Delimiters

Delimiters are special characters used to separate or group code elements:

• Parentheses: () - used for function calls and grouping
• Brackets: [] - used for arrays
• Braces: {} - used for blocks of code
• Semicolon: ; - used to end statements
• Comma: , - used to separate items

Logic science

Propositional Logic

Propositional logic is the foundation of logical reasoning, which forms the basis for all programming. Propositions are simple statements that can either be true or false. In programming, we represent these as boolean values. A proposition is atomic if it cannot be broken down further.

Examples of propositions:

• A = "The sky is blue" (true)
• B = "2 + 2 = 5" (false)
• C = "Today is Friday" (depends on the day)

Truth Tables

A truth table is a mathematical table that shows all possible combinations of truth values for one or more propositions and the resulting truth value for a logical operation.

AND Operator (∧): The result is true only if both propositions are true.

OR Operator (∨): The result is true if at least one proposition is true.

NOT Operator (¬): The result is the opposite of the proposition.

Boolean algebra

Boolean algebra is a system of logic in which variables may take on one of two values: 0 (FALSE) or 1 (TRUE). In programming, we use boolean variables and operations to control program flow.

Boolean Laws

• Identity: A ∧ T = A; A ∨ F = A
• Null: A ∧ F = F; A ∨ T = T
• Idempotent: A ∧ A = A; A ∨ A = A
• De Morgan's: ¬(A ∧ B) = ¬A ∨ ¬B; ¬(A ∨ B) = ¬A ∧ ¬B

Predicate Logic

Predicate logic extends propositional logic by using predicates, which are functions that return a boolean value. A predicate can have arguments and can express properties or relationships between elements.

Examples:

• P(x) = "x is greater than 5"
• R(x, y) = "x is related to y"

Sets and Domains

In predicate logic, we work with sets and domains of elements. A set is a collection of unique elements. We can represent sets using standard mathematical notation:

• A = {a, b, c}
• N = {1,2,3}

Relations

Elements in a datasets can be associated using a relation. A relation between two arguments is called binary relation. The elements of a binary relation is the Cartesian product (x). A relation R is a predicate with (at least) two arguments:

R = A x N = {(a,1), (a,2), (a,3), (b,1), (b,2), (b,3), (c,1), (c,2), (c,3)}

In logic, sometimes we use symbol "∼" to define relations: For example: x∼y means x is related to y. Also we can say that pair (x,y) belong to relation "∼". Most of the time a relation is represented by a binary operator. For example we can use { =, <, > } to express the relation between numbers.

Quantifiers

In logic we use quantifiers to refer to a set of elements. The quantifier is a symbol that specify one or more elements we are referring to. So far I have not found programming languages that are using quantifiers. Usually a quantifier is implemented using one or more statements.

• ∀ = all (universal quantifier)
• ∃ = exist (existential quantifier)

In programming, the exist "∃" can be implemented by searching for a particular element in a data set. If the element satisfy the relation the predicate is True, otherwise False.

Belonging

We can express if an element belong or does not belong to a particular domain or dataset. This is very helpful to establish meaningful logic expressions involving data sets.

• ∈ means: "belong"
• ∉ means: "does not belong"

Sometimes we factor out a qualifier to refer to more than one element. For example the expression: ∃(x,y,z) ∈ {0,1,2,3,4,5,6,7,8,9}, it means exist a pair of 3 single digit numbers.

Identity

Identity is an equivalence relation. We have seen this symbol before, used for propositional logic: { "=" , "≡"}. This symbol is polymorphic. It can be used with any kind of elements not only propositions but also predicates or terms. It's because identity relation is symmetric, transitive and reflexive.

Properties

Understanding relations requires knowing their properties. The following three properties are especially important in logic and mathematics:

• Reflexive: A relation '∼' on a set A is reflexive if every element is related to itself.
  ∀a ∈ A, a ∼ a
• Symmetric: A relation '∼' on a set A is symmetric if the order of the elements does not matter. If a is related to b, then b is related to a.
  ∀a, b ∈ A, (a ∼ b) → (b ∼ a)
• Transitive: A relation '∼' on a set A is transitive if a direct relationship between the first and second element and the second and third element implies a relationship between the first and third.
  ∀a, b, c ∈ A, ((a ∼ b) ∧ (b ∼ c)) → (a ∼ c)

An "equivalence relation" is a relation that is reflexive, symmetric, and transitive. The equality (=) operator is a common example of an equivalence relation.

Order

An set or range of elements is usually ordered. When a set is ordered you can apply special relations between consecutive elements.

For example we can study numbers and relation between numbers using predicate logic. A number can be greater, equal or less than other number. Also a number can be different or not equal to other number. In the next table we show you the most usual notation for comparison operators.

Lists & Sets

To understand the predicate logic better we need to grasp a sets and collections of things. A set is a group of things that are unique represented and not duplicated. All things in a set can be similar or can have characteristics in common. We call members of a set: "elements".

Examples of 2 Sets

Sometimes we need to make a collection of elements that can be duplicated. In this case we have a "list of elements" not a sets because some of the elements are not unique. We refer in computer science to a list or a set of element by using the term: collection.

Predicate logic study the elements of collections and the relations between them. For example an element can belong to a set or do not belong to a set. Two sets can contain the same elements or different elements. Then the sets are equal or not equal.

Operations

Operations between sets can produce new sets or logic results. Also it is possible to make operations between one set and one value to check if the value belong to a set. In next table we use Unicode symbols for operators between sets:

Intersection

Set Intersection

Union

Set Union

Difference

Set Difference

Vector & Matrix

In mathematics there are 2 significant numeric collections: Vectors and Matrices. In Python we define a "List" that can have one or more dimensions and can hold a Vector or a Matrix. Some languages use term: "Array". Sometimes "Array" and "List" are different things with similar properties.

Fuzzy Logic

The theory of Quantum Computing is using a similar concept called "Quantum Logic". This is based on quantum bits (qbits). The difference is that number of states between {0,1} in quantum computing is limited while in fuzzy logic the number of states between {0,1} are infinite.

State Machines

Moore machines

Moore machine was invented by Edward Moore in 1956 and is called Moore machine. Moore machines consist of states and transitions. States are able to produce outputs, and the output is determined solely by the current state, not by any input.

Mealy machines

Mealy machines were invented by George H. Mealy in 1955. In comparison with Moore machines, Mealy machines produce outputs only on transitions and not in states. This often results in state diagrams with fewer states because more logic can be put on transitions.

Harel state charts

A state chart is a visual formalism for complex systems. Basically Harel state charts are Mealy/Moore state machines extended by further concepts that allow us to model complex systems in a practical way.

David Harel did his PhD in 1978 at the Massachusetts Institute of Technology in Cambridge. He then became a Professor for Computer Science at the Weizmann Institute in Jerusalem in 1980.

David Harel:  A complex system cannot be beneficially described in this naive fashion, because of the unmanageable, exponentially growing multitude of states, all of which have to be arranged in a flat non stratified fashion, resulting in an unstructured, unrealistic, and chaotic state diagram.

UML state-chart

UML stands for "Unified Modeling Language". UML state machines have the characteristics of both Mealy machines and Moore machines.UML state machine is an object-based variant of Harel state chart.

Usability

State machines are useful to analyze and theorize the computation using abstract notations. In practice there are efforts to create various code generators to translate the state charts into source code.

Representation

State machines can be represented in form of tables and diagrams. States are represented in diagrams by round circles, or shapes and transitions between states by arc of circles with arrow at end. A state machine diagram is usually oriented from top down or from left to right. The entry point is usually represented by a black dot. Some states are marked as "accepting" states with double line circle.

State Machine

Turing Machine

Separately, during World War II, Alan Turing designed an electro-mechanical machine called the Bombe, which was instrumental in breaking the German Enigma code. However, the Bombe and the theoretical Turing Machine are not the same thing.

Today, we use the term "Turing Complete" to classify a programming language or system that has a computational power equivalent to a universal Turing machine. Theoretically, such a system can solve any problem that a computer can solve, given enough time and memory. A Turing machine can be represented as a finite state machine, but a conceptual diagram is often easier to grasp:

Turing Machine

Artificial Intelligence

Machine Learning

While true artificial intelligence has not yet been achieved, we have created machine learning, a field of AI that uses statistical techniques to enable computer systems to "learn" from data. It is built on the idea that by analyzing vast amounts of data, a system can identify trends and patterns, which can be used to make predictions or recognize new patterns, thereby simulating a form of intelligence.

Example:

Statistic Chart
Linear Trend

Neural Network

Neural networks are a key component of machine learning, in which a computer learns to perform a task by analyzing training examples. Typically, these examples have been hand-labeled in advance.

Most modern neural networks are organized into layers of nodes and are "feed-forward," meaning data moves through them in only one direction. An individual node might be connected to several nodes in the layer from which it receives data and several nodes in the layer to which it sends data.

Neural Network
Conceptual Representation

The concept of neural networks was first proposed in 1944 by Warren McCullough and Walter Pitts, two University of Chicago researchers who later moved to MIT in 1952 as founding members of what is sometimes called the first cognitive science department.

DEI in AI

DEI in AI stands for diversity, equity, and inclusion. It is the practice of ensuring that AI systems are fair and inclusive, and that they do not discriminate against people based on their race, gender, religion, or other protected characteristics.

DEI in AI is important because AI systems are increasingly being used to make decisions that affect people's lives, such as whether they get a job or a loan. If these systems are biased, they can perpetuate discrimination and inequality.

There are a number of ways to promote DEI in AI, such as:

• Using diverse data sets to train AI systems.
• Developing AI systems that are fair and unbiased.
• Training AI developers on DEI principles.
• Monitoring AI systems for bias and discrimination.
• Taking steps to mitigate bias and discrimination in AI systems.

DEI in AI is a complex and challenging issue, but it is important to address it in order to ensure that AI systems are fair and inclusive.

What is LLM?

LLM stands for **Large Language Model**. It is a type of artificial intelligence (AI) that is trained on a massive amount of text data. This data can include books, articles, code, and other forms of written language. LLMs are able to learn the patterns and relationships between words and phrases in this data, which allows them to generate new text that is similar to the text they were trained on.

LLMs are used in a variety of applications, including:

• Machine translation: LLMs can be used to translate text from one language to another. For example, Google Translate uses an LLM to translate text between over 100 languages.
• Text summarization: LLMs can be used to summarize long pieces of text into shorter summaries. This can be useful for quickly getting the main points of a document or article.
• Chatbots: LLMs can be used to create chatbots that can have natural conversations with humans. This can be used for customer service applications, or for providing information or assistance to users.
• Creative writing: LLMs can be used to generate creative text formats, like poems, code, scripts, musical pieces, email, letters, etc. This can be used for entertainment purposes, or for generating new ideas.

LLMs are still under development, but they have already learned to perform many kinds of tasks. As LLMs continue to develop, they are likely to become even more powerful and versatile.

Here are some examples of LLMs:

• GPT-3: GPT-3 is an LLM developed by OpenAI. It has 175 billion parameters and is one of the most powerful LLMs in the world. GPT-3 can be used for a variety of tasks, including machine translation, text summarization, and creative writing.
• LaMDA: LaMDA is an LLM developed by Google AI. It is similar to GPT-3 in terms of its capabilities, but it has been specifically designed to be more factual and less likely to generate offensive or harmful content.
• Megatron-Turing NLG: Megatron-Turing NLG is an LLM developed by Google AI and NVIDIA. It is the largest LLM in the world, with 530 billion parameters. Megatron-Turing NLG is still under development, but it has already shown promising results in machine translation and text summarization.

LLMs are a powerful new tool that has the potential to revolutionize the way we interact with computers. As LLMs continue to develop, they are likely to become even more important in our lives.

Generative AI

Generative AI is a branch of artificial intelligence (AI) that focuses on creating new content, such as text, images, audio, and code. It is able to learn the patterns and structures of existing data and then use this knowledge to generate new data that is similar in style or form. This makes it a powerful tool for a variety of applications, including art, design, and product development.

Disclaimer

This website was improved using AI. You can use AI for free using Google's web application. All you need is a Google account. This service is in public beta. While AI can assist with programming, its suggestions should be carefully reviewed for correctness and security.

Google AI: Gemini