engineering<--

Programming Concepts

This article is designed for beginners in computer science. We start with basic concepts used in computer science. All programming languages are based on these concepts. Some of these are used to make new software and some others for resolving problems or running a business.

Computer programs

Computer programs are step by step instructions that a computer can execute. These are grouped in statements that are stored into a modules or package. A program can have many packages. Sometimes programs are also called applications.

Application purpose

Applications can be created using one or more programming languages. Usually an application is designed to resolve a specific problem. Most applications have multiple features that you can use to send or receive information to and from the application using input/output devices like a console (display window) or a printer.

Application execution

Program execution will resolve a problem and communicate the result or will have a physical effect: for example printing a message to the computer monitor or moving the arm of a robot. A program can be executed one or multiple times.

Code base

Computer programs consist of files, stored in folders. We call this a "project" or code base or code source. A project contains text files having language specific extension, data files and perhaps images, audio and video files. Usually one file is the main file that uses the other files. A project should contain also documentation files.

Sub-programs

Sub-programs have the role: "separation of concerns". It means one sub-program does only one small thing. The main program combine the effects of multiple sub-programs to implement the end-to-end functionality of the system. Sub-programs are contained in programs or delivered as components or libraries to be re-used in many applications.

Depending on the computer language sub-programs are called: sub-routines, procedures, functions, methods or rules. Common is that all encapsulate a specific functionality. The main program is orchestrating the execution of sub-programs by transferring the execution main process into sub-processes that can be executed synchronously or asynchronously (sometimes in parallel).

Formal parameters

Sub-programs can receive parameters and can compute results. Sometimes we call these "formal parameters" or "receiver variables". Usually parameters for sub-programs are immutable during sub-program execution.

Parameters are similar to local variables. They are known only inside the sub-routine. Once the subroutine is finished, the value of the parameters is lost or can be propagated back to the caller. This depends on the declaration syntax and purpose of the parameter.

Believe it or not in Fortran, parameters are not the same thing as in other languages. In fortran, parameters are actually constants. In most other languages parameters are variables.

Programming Symbols

A program is made of symbols that are used to create expressions. Most languages are using expressions that are similar to the ones we learn in mathematics. Sometimes beginners have hard time understanding the difference between "expressions", "symbols" and relation with "functions". We will explain these things here.

Symbols can be single character or multi-character. Single character symbols can be {"digits", "letters", "ASCII", "UNICODE"}. We use symbols to create: operators, punctuation marks, numeric literals, string literals, identifiers, enumerations and data collections.

In general a program manipulate "data". This is also made of symbols. We learn in school to count using Arabic digital symbols: 0,1,2,3,4,5,6,7,8,9. We also learn how to read and write text using Latin alphabet: Aa, Bb, Cc, Dd, Ee, Ff, Gg ... Zz. This is not the case in all cultures around the globe. Some cultures are using other symbols to represent data.

Encoding

The way we represent data using the symbols is called encoding. The most popular encoding system known as Unicode. Before Unicode was invented we used different encodings system known as: ASCII. For different countries there is an Extended ASCII table that has diacritics and special characters. Also Unicode have several variations: UTF16, UTF32 and UTF8.

Binary

Computer is using binary representations of "1" and "0" for encoding symbols. The "1" is usually a positive electrical charge while "0" represents no charge. It is similar to a light bulb. When light is on we can consider it logic = "1". When lights is off we consider logic="0″.

Storage

Electronic devices can store data using specialized devices to store data. There are several physical methods: "electro-static", "electro-magnetic", "optic" or "magneto-optic" and of course "solid state - electronics". The point is, all computation is digital, and is based on a binary system.

Logic Science

The logic was discovered by Greeks and formalized by Aristotle. Logic means thought or reason. Initially the logic was described only by philosophy. However in time a new mathematical branch was developed. Logic is the science of truth. The purpose of logic is wisdom.

Fundamental Concepts

Before you can understand logic we must define some words and rules that are well established and can’t be invented by yourself. Many software developers do not understand the definitions and therefore can’t have meaningful conversations and arguments. Read the entire article slowly to learn these things before you can say you know logic.

Inference

Inference is the most important concept in logic. It is also known as deduction or implication. This is a conclusion we reach after we study the arguments or premises into a sentence. Inference is a way of thinking and reasoning.

Logical inference: Using valid arguments into a sentence we can derive a logical conclusion. Sound arguments will produce the same conclusion no matter who makes the arguments or who derive the conclusion.

The inference and reason do not work the same in all cultures. For example a Christian will not reach the same conclusion from a religious sentence as another person that is Buddhist or Muslim. The words may be interpreted in different ways and lead to different conclusions.

Sentences

Sentences are enumerations of words that together express an idea, describe a situation or fact, give a command or inquiry about a fact, status or situation. Sentences can be classified in four categories

Imperative sentences (orders/commands)
Interrogative sentences (inquiries)
Declarative sentences (statements)
Exclamation sentences (surprise)

Elements

A sentence can have 5 elements: { subject, verb, object, complement, and adjunct } (SVOCA). The subject is the performer of an action or the agent of the verb. It is usually at the beginning of a sentence, and it is generated by a noun or any of its equivalents, such as a pronoun, a noun phrase, or a noun clause. Sentences are studied by the grammar of a language.

Statements

Statements: are declarative sentences that express a fact, idea or opinion. Statements do not make requests, give commands or express a surprise. A statement usually end with dot (.)

Some statements are neither true neither false but have an undetermined value that can be “maybe” or “probable”. There is a special logic for this kind of statements.

Propositions

Propositions: are statements that can be true or false. A proposition has a single value, is either true, either false but not both in the same time. Propositions are used in propositional logic.

There are examples of declarative sentences that are not propositions. For example, “This sentence is false” is not a proposition, since it can’t be true. For instance, if we assign it the truth value True, then we are saying that “This sentence is false” is in fact true but the sentence is false by definition so it can’t be sometimes true.

Fundamental principle

It is important to know that logic can't create good inference from imperative or interrogative sentences but only from declarative sentences. Imperative sentence is also a command from a superior entity or authority like Emperor, Queen, Fuhrer, President or God.

Info: By making inference from imperative sentences one culture can fall into irrational believes, wrong lows and social injustice. Logic can only work based on facts and solid arguments not on irrational commands.

Composition rules

Statements can be simple or composite. Composite statements can include arguments. A composite statement is made of simple statements. Most statements are true or false.

Let's say we have an argument or a premise and a sentence. Following are some logic rules that can apply to these two:

Identity rule: An argument is identical with itself and not something else;
Contradiction rule: An argument cannot be true and false in same sentence;
Validity rule: An argument is valid if the truth of premises lead to truth of the conclusion.

Validity and Soundness

A deductive argument is said to be valid if takes a form that makes it impossible for the premises to be true and the conclusion nevertheless to be false. Otherwise, a deductive argument is said to be invalid.

A deductive argument is sound if and only if it is both valid, and all of its premises are true. Otherwise, a deductive argument is unsound. Unsound arguments are invalid.

Propositional Logic

The propositional logic study the relation between arguments and sentences. Deductive arguments can be propositions or sentences that can have Boolean values: True or False. This logic do not deal with the content of propositions but only with the relation between propositions.

In this rules we use A and B and P to represent logical propositions. Part of propositional logic are relations: IS, AND, OR, NOT. This relations have very precise rules that makes propositional logic a reliable theory.

Fundamental rules:

There are several Logical rules that are universal and can't be broken. Nobody can contradict them in any culture or language. If you understand propositional logic rules you have done your first step toward understanding computer science.

If A and B are true then P = A and B is true.
If A and B are false then P = A and B is false.
If A or B is false then P = A and B is false.
If A or B is true then P = A or B is true.
If A is true then P = not A is false.
If B is false then P = not B is true.

In addition operators "or" "and" are commutative. That means:

A and B == B and A
A or B == B or A

Morgan's laws:

Sometimes you can transform one logical expression into another equivalent expression without knowing the value of the arguments. These rules are very useful in Computer Science to optimize Boolean expressions.

the negation of a disjunction is the conjunction of the negations;
the negation of a conjunction is the disjunction of the negations;

Let's explain what it means in terms of expressions:

not ( not A) == A
not (A or B) == (not A) and (not B)
not (A and B) == (not A) or (not B)

Boolean algebra

Boolean algebra is the branch of algebra in which the values of the variables are the truth values true and false, usually denoted 1 and 0 respectively. This contrasts with general algebra that study other numbers not only the value 1 and 0.

Logical Operations:

I have found these are the most suitable symbols for logical operators. Though no programming language yet uses them.

symbol	alternative	meaning	notes
¬	!	NOT	negation
∧	&	AND	conjunction
∨	\|	OR	disjunction
⊕	^	XOR	exclusive disjunction
↓		NOR	p ↓ q = ¬ (p ∨ q)
↑		NAND	p ↑ q = ¬ (p ∧ q)

The table of truth:

Next table shows all possible combinations and the result for logical operators.

p	q	¬ p	¬ q	p ⊕ q	p ∧ q	p ∨ q
1	1	0	0	0	1	1
1	0	0	1	1	0	1
0	1	1	0	1	0	1
0	0	1	1	0	0	0

Predicate logic

Predicate logic is an extension of propositional logic. In propositional logic we look at propositions and arguments. That can be true or false but in reality there are many other things that are not true or false. In predicate logic we use terms and relations between terms to establish the value of truth of a proposition. Predicate logic is essential in programming, to deal with numbers and data sets.

Use case

Predicate logic define new terminology and notations. Understanding this terminology will help you read complex documentation used in programming. This may be a language specification or high level requirement that you can receive on the job. You can be invited to elaborate the requirements in detailed design that require logic.

Predicate

A predicate is a property or a relation between one or more terms. One term can be a variable or a constant. A predicate is not a logic entity but it can be evaluated to value True, False or invalid, depending on the range of predicate arguments.

Predicates with one argument, are called monadic. Predicates with more arguments are called dyadic, n-adic or poly-adic. Some languages are using monadic predicates while other languages can accept poly-adic predicates.

Arguments

The arguments are specific values, used to evaluate a predicate. Sometimes the arguments are called: predicate variables or parameters. In programming and mathematics a predicate is a function or an expression that can be evaluated to True or False depending on the arguments.

Expressions

Simplest predicates are the ones expressing properties of things. We use uppercase letters to express a predicate and lowercase letters to express arguments. Some examples of predicates:

A(x): x is tall
B(x,y): x owes money to y
C(x,y,z): x borrowed y from z

Domain

A variable used in a predicate can be bound to a specific domain. When a variable is not bound, is called “free variable”. This is not an argument but it can influence the result of predicate. In logic papers you may encounter the name “universe of discourse” (UD) that is actually a domain or range of values from a particular data set.

Data set

A data set is a range of values. It can be numeric or can represent symbols or objects. A data set can be empty, or can have one or more elements. Some data sets can have an infinite number of elements. In a data-set the elements are unique. The elements in a data-set are usually ordered. A data set is represented as an enumeration of symbols separated by comma and enclosed in brackets, like:

A = {a, b, c}
N = {1,2,3}

Relations

Elements in a datasets can be associated using a relation. A relation between two arguments is called binary relation. The elements of a binary relation is the Cartesian product (x). A relation R is a predicate with (at least) two arguments:

R = A x N = {(a,1), (a,2), (a,3), (b,1), (b,2), (b,3), (c,1), (c,2), (c,3)}

In logic, sometimes we use symbol “∼” to define relations: For example: x∼y means x is related to y. Also we can say that pair (x,y) belong to relation “∼”. Most of the time a relation is represented by a binary operator. For example we can use { =, <, > } to express the relation between numbers.

Quantifiers

In logic we use quantifiers to refer to a set of elements. The quantifier is a symbol that specify one or more elements we are referring to. So far I have not found programming languages that are using quantifiers. Usually a quantifier is implemented using one or more statements.

∀ = all (universal quantifier)
∃ = exist (existential quantifier)

Note: Qualifiers can be combined together to create meaningful predicate expressions. Most common we can use for all elements in domain there is or exist x that satisfy a relation R(x).

In programming, the exist "∃" can be implemented by searching for a particular element in a data set. If the element satisfy the relation the predicate is True, otherwise False.

Belonging

We can express if an element belong or does not belong to a particular domain or dataset. This is very helpful to establish meaningful logic expressions involving data sets.

∈ means: “belong”
∉ means: “does not belong”

Sometimes we factor out a qualifier to refer to more than one element. For example the expression: ∃(x,y,z) ∈ {0,1,2,3,4,5,6,7,8,9}, it means exist a pair of 3 single digit numbers.

Note: In programming we can refer to a range of numbers using square brackets. [x-y] or [x..y] or [x: y]. This convention is not the same in all languages. You must learn the specific syntax.

Identity

Identity is an equivalence relation. We have seen this symbol before, used for propositional logic: { “=” , “≡”}. This symbol is polymorphic. It can be used with any kind of elements not only propositions but also predicates or terms. It’s because identity relation is symmetric, transitive and reflexive.

Properties

Understanding relations, assume to know the symbol definition, and the properties of specific relations. Next 3 properties are very important:

relation ”∼” is symmetric: ∀(a,b) ∈ R*(a* ∼ b) \=> (b ∼ a)
relation ”∼” is transitive: ∀(a) ∈ R*(a* ∼ a)
relation ”∼” is reflexive: **∀(a, b, c) ∈ R((a∼b) ∧ (b∼c)) => (a ∼ c)

Order

An set or range of elements is usually ordered. When a set is ordered you can apply special relations between consecutive elements.

For example we can study numbers and relation between numbers using predicate logic. A number can be greater, equal or less than other number. Also a number can be different or not equal to other number. In the next table we show you the most usual notation for comparison operators.

Math	CS	Description	True = 1	False = 0
=	==	Equal	1 == 1	1 == 0
≠	!=	Not equal	1 != 0	1 != 1
>	>	Greater than	2 > 1	5 > 5 + 1
<	<	Less than	0 < 1	1 < 0
≥	>=	Greater than or equal to	1 >= 0	1 >= 2
≤	<=	Less than or equal to	1 <= 1	1 <= 0

Lists & Sets

To understand the predicate logic better we need to grasp a sets and collections of things. A set is a group of things that are unique represented and not duplicated. All things in a set can be similar or can have characteristics in common. We call members of a set: "elements".

Examples of 2 Sets

Sometimes we need to make a collection of elements that can be duplicated. In this case we have a "list of elements" not a sets because some of the elements are not unique. We refer in computer science to a list or a set of element by using the term: collection.

Predicate logic study the elements of collections and the relations between them. For example an element can belong to a set or do not belong to a set. Two sets can contain the same elements or different elements. Then the sets are equal or not equal.

Operations

Operations between sets can produce new sets or logic results. Also it is possible to make operations between one set and one value to check if the value belong to a set. In next table we use Unicode symbols for operators between sets:

symbol	example	meaning
∩	R = A ∩ B	Intersection between two sets => new set
∪	R = A ∪ B	Union between two sets => new set
⊂	b = A ⊂ B	Set is A included in superset B: => boolean
⊃	b = A ⊃ B	Set A contain subset B: => boolean
Δ	Δ = A - B	Set difference, => new set
∈	b = x ∈ A	Belong: check if element belong to collection => boolean
≡	b = A ≡ B	Equivalent: check if A has same elements as B => boolean
∀	∀ x ∈ A	Any element: used in collection qualification => boolean
∃	∃ x ∈ A	Exist: used in collection qualification => boolean

Intersection

Intersection between two sets A, B will be a smaller set R that will contain all common elements of A and B. All other elements will not be included.

Set Intersection

Union

Union between two sets A, B will be a larger set R that contain all elements of A and all elements of B, but duplicate elements will be included only once.

Set Union

Difference

Difference between two sets A, B will be a set C that contain all elements of A but not elements that are common with B. There is a second difference D that we can make between B and A.

Set Difference

Vector & Matrix

In mathematics there are 2 significant numeric collections: Vectors and Matrices. In Python we define a "List" that can have one or more dimensions and can hold a Vector or a Matrix. Some languages use term: "Array". Sometimes "Array" and "List" are different things with similar properties.

Fuzzy Logic

Fuzzy logic is an attempt to incorporate nature's inherent fuzziness into technology. This logic works not only with values 1=True and 0=False but we some values in between. Not everything is black or white but there is a gray area. This may be related to probabilistic values that are values between 0 and 1. Fuzzy Logic is used in neural networks and machine learning.

The theory of Quantum Computing is using a similar concept called "Quantum Logic". This is based on quantum bits (qbits). The difference is that number of states between {0,1} in quantum computing is limited while in fuzzy logic the number of states between {0,1} are infinite.

State Machines

A "state machine", is a mathematical model of computation. It is an abstract machine that can be in exactly one of a finite number of states at any given time. We call this also a "Finite State Machine" FSM.

Moore machines

Moore machine was invented by Edward Moore in 1956 and is called Moore machine. Moore machines consist of states and transitions. States are able to produce outputs, and the output is determined solely by the current state, not by any input.

Mealy machines

Mealy machines were invented by George H. Mealy in 1955. In comparison with Moore machines, Mealy machines produce outputs only on transitions and not in states. This often results in state diagrams with fewer states because more logic can be put on transitions.

Harel state charts

A state chart is a visual formalism for complex systems. Basically Harel state charts are Mealy/Moore state machines extended by further concepts that allow us to model complex systems in a practical way.

David Harel did his PhD in 1978 at the Massachusetts Institute of Technology in Cambridge. He then became a Professor for Computer Science at the Weizmann Institute in Jerusalem in 1980.

David Harel: A complex system cannot be beneficially described in this naive fashion, because of the unmanageable, exponentially growing multitude of states, all of which have to be arranged in a flat non stratified fashion, resulting in an unstructured, unrealistic, and chaotic state diagram.

UML state-chart

UML stands for "Unified Modeling Language". UML state machines have the characteristics of both Mealy machines and Moore machines.UML state machine is an object-based variant of Harel state chart.

Usability

State machines are useful to analyze and theorize the computation using abstract notations. In practice there are efforts to create various code generators to translate the state charts into source code.

Representation

State machines can be represented in form of tables and diagrams. States are represented in diagrams by round circles, or shapes and transitions between states by arc of circles with arrow at end. A state machine diagram is usually oriented from top down or from left to right. The entry point is usually represented by a black dot. Some states are marked as "accepting" states with double line circle.

State Machine

Turing Machine

A Turing Machine was invented by Alan Turing to break the German encryption machine Enigma. Enigma code was broken using the first universal computation machine that was named later computer.

Today we use term: Turing Complete to classify a computer language that can do what a Turing Machine can do. Theoretical it can resolve any finite logical problem, by splitting it into simple operations. A Turing machine can be represented as a finite state machine but we show instead a conceptual diagram that is more easy to grasp:

Turing Machine

Artificial Intelligence

Artificial intelligence is often used to describe machines (or computers) that mimic cognitive functions usually associated with the human mind, such as learning and reasoning.

Machine Learning

We have not yet achieved artificial intelligence per se but we have created instead something called machine learning that is a fancy word for statistic. It is build on the idea that we can analyze data and predict new data by detecting a "trend". This can be potentially used to predict the future data or to recognize "patterns" therefore simulate a primitive intelligence.

Example:

statistic chart
Linear Trend

Neural Network

Neural nets are a means of doing machine learning, in which a computer learns to perform some task by analyzing training examples. Usually, the examples have been hand-labeled in advance.

Most of today neural nets are organized into layers of nodes, and they use feed-forward, meaning that data moves through them in only one direction. An individual node might be connected to several nodes, in layer from which it receives data, and several nodes in the layer to which it sends data.

Neural Network
Conceptual Representation

Neural networks were first proposed in 1944 by Warren McCullough and Walter Pitts, two University of Chicago researchers who moved to MIT in 1952 as founding members of what's sometimes called the first cognitive science department.

DEI in AI

DEI in AI stands for diversity, equity, and inclusion. It is the practice of ensuring that AI systems are fair and inclusive, and that they do not discriminate against people based on their race, gender, religion, or other protected characteristics.

DEI in AI is important because AI systems are increasingly being used to make decisions that affect people's lives, such as whether they get a job or a loan. If these systems are biased, they can perpetuate discrimination and inequality.

There are a number of ways to promote DEI in AI, such as:

Using diverse data sets to train AI systems.
Developing AI systems that are fair and unbiased.
Training AI developers on DEI principles.
Monitoring AI systems for bias and discrimination.
Taking steps to mitigate bias and discrimination in AI systems.

DEI in AI is a complex and challenging issue, but it is important to address it in order to ensure that AI systems are fair and inclusive.

What is LLM?

LLM stands for **Large Language Model**. It is a type of artificial intelligence (AI) that is trained on a massive amount of text data. This data can include books, articles, code, and other forms of written language. LLMs are able to learn the patterns and relationships between words and phrases in this data, which allows them to generate new text that is similar to the text they were trained on.

LLMs are used in a variety of applications, including:

Machine translation: LLMs can be used to translate text from one language to another. For example, Google Translate uses an LLM to translate text between over 100 languages.
Text summarization: LLMs can be used to summarize long pieces of text into shorter summaries. This can be useful for quickly getting the main points of a document or article.
Chatbots: LLMs can be used to create chatbots that can have natural conversations with humans. This can be used for customer service applications, or for providing information or assistance to users.
Creative writing: LLMs can be used to generate creative text formats, like poems, code, scripts, musical pieces, email, letters, etc. This can be used for entertainment purposes, or for generating new ideas.

LLMs are still under development, but they have already learned to perform many kinds of tasks. As LLMs continue to develop, they are likely to become even more powerful and versatile.

Here are some examples of LLMs:

GPT-3: GPT-3 is an LLM developed by OpenAI. It has 175 billion parameters and is one of the most powerful LLMs in the world. GPT-3 can be used for a variety of tasks, including machine translation, text summarization, and creative writing.
LaMDA: LaMDA is an LLM developed by Google AI. It is similar to GPT-3 in terms of its capabilities, but it has been specifically designed to be more factual and less likely to generate offensive or harmful content.
Megatron-Turing NLG: Megatron-Turing NLG is an LLM developed by Google AI and NVIDIA. It is the largest LLM in the world, with 530 billion parameters. Megatron-Turing NLG is still under development, but it has already shown promising results in machine translation and text summarization.

LLMs are a powerful new tool that has the potential to revolutionize the way we interact with computers. As LLMs continue to develop, they are likely to become even more important in our lives.

Generative AI

Generative AI is a branch of artificial intelligence (AI) that focuses on creating new content, such as text, images, audio, and code. It is able to learn the patterns and structures of existing data and then use this knowledge to generate new data that is similar in style or form. This makes it a powerful tool for a variety of applications, including art, design, and product development.

Disclaim

This website was improved using AI. You can use AI for free using Google web application. All you need is a Google account. This service is in beta. It can not be used for programming, but if you ask the right questions you will get good responses.

Google AI: bard (experimental)

Warning: AI can do mistakes. It generate 3 responses. You can select and vote which one is better. AI pretend to learn from you. If you use AI generated content, you should verify and validate the response.

Read next: Numeric Algebra