Certificação 70-483 - Programming in C#

26 Jan 2019

Recentemente andei estudando para o exame 70-483 da Microsoft. E como faço várias anotações resolvi agrupá-las e publicar para quem possa interessar.

Minha leitura foi baseada no livro Exam Ref 70-483: Programming in C# autorado pelo Wouter de Kort.

Em alguns pontos é possível ver um !!!. Isto quer dizer que a explicação do livro foi breve demais e preciso pesquisar mais a fundo.

Obs: As anotações estão em inglês, como o livro estava em inglês, decidi anotar na mesma língua para não ficar mudando de contexto (e até para praticar o inglês).

Chapter 1 - Manage program flow

Objective 1.1 - Implement multithreading and asynchronous processing

• Threading
  • Distribute the processing throught many CPU cores
  • Also called parallelism
  • Windows manage the CPU time of all threads to make sure that all threads executes
  • To switch threads, the SO needs to save CPU registers and other state data to restore it later
    • Too many thread switch leads to performance issues?
• The Thread class enable creation, management and peeking the thread status
  • It should be used in special situations
• Synchronization ensures that two threads do not execute some code at the same time
• Foreground threads are threads that keep the application alive
  • When all foreground threads are dead, the background threads are also killed
• Stopping threads
  • The method Stop can stop a thread
    • But a ThreadAbortedException is thrown
  • Is recommended to use a semaphore to stop threads
    • Or cancellation tokens
• The ThreadStatic attribute allows each thread to get a copy of a static field
• To use local thread data and initialize the field for each thread use the ThreadLocal<T> type
• The thread context is accessed by the Thread.CurrentThread property
  • Thread id, thread's CultureInfo, thread's principal, priority, etc
• Thread pools
  • Thread creation can cost time and resources
  • It's possible to queue work items to be executed by the next available thread from the pool
  • ThreadPool.QueueWorkItem
  • But there are limited number of threads
  • Because threads are being reused, they also reuse the local state
    • Some Thread properties like Name, Principal, Culture, Priority, etc
    • On the ThreadPool documentation has a section about this
    • There is also an issue about it (still open today, 10/02/2019)
• Tasks
  • Useful to know if the operation was successfull and also the get a return value
  • By default, Tasks uses threads from the ThreadPool
  • The Task.Wait method is equivalent to the Thread.Join method
  • Reading Task.Result property will make the caller thread to wait until the tasks finishes
  • Tasks can be combined using the ContinueWith method (documentation)
    • The same idea of the Javascript callbacks
    • Continuation Tasks can be run on specific contitions usint the TaskContinuationOptions flag
      • Options like OnlyOnCanceled that runs only when the previous Task is cancelled
      • Other options like OnlyOnFaulted or OnlyOnRanToCompletion
      • Documentation
  • ChildTasks
    • Tasks that are attached to parents Tasks
    • Parent tasks only finishes when all child tasks are ready
    • TaskCreationOptions.AttachedToParent flag
    • The child task is synchronized with the parent task
      • The parent task only finishes if the child finishes
    • Documentation
  • TaskFactory
    • Useful to avoid task configuration flags when creating many Tasks
    • Create an instance of TaskFactory passing the desired TaskCreationOptions and TaskContinuationOptions flags
      • Use the StartNew methods to spawn Tasks with the previously provided configuration
  • Waiting for Tasks
    • It's possible to use the Task.Wait to wait for a single Task
    • When waiting for multiple Tasks, use the Task.WaitAll
    • Task.WaitAny waits for the first finished Task
    • To schedule a continuation method, use Task.WhenAll
• Parallel class
  • Tasks that are executed concurrently
  • Smaller work or synchronization to access resources can hurt performance
  • Loops
    • Parallel.For
    • Parallel.ForEach
  • ParallelLoopState (last parameter of the loop methods) can control the iterations
    • Stop terminates everything
    • Break terminates only the current operation
  • ParallelLoopResult
    • Contains the status o the loop (IsCompleted property)
    • Contains the number of the last iteration (LowestBreakIteration property)
  • !!!
    • The caller thread is blocked?
    • It uses ThreadPool internally? Task?
    • More examples
• async and await
  • Mark the method as async to let the compiler turn your code into a state machine and simplify asynchronous operations
  • The await keyword offloads the operation to another thread
  • It improves the responsiveness, not performance!
  • Synchronization context
    • Connects the app model to its threading model
      • UI thread and background thread
    • await saves the current synchronization context and restores it back when the Task finishes
    • When the context synchronization is not needed, use the ConfigureAwait(false) methods of the Task class
      • Then, the rest of the code runs on any thread
  • Avoid the Task<Void> type because it's not possible to inspect exceptions
    • Except when its an event handler !!!
  • Avoit async methods without await calls
• PLINQ
  • Can be used to turn sequential queries into parallel ones
  • PLINQ extension methods are defined in System.Linq.ParallelEnumerable
  • Parallel queries can be created withe AsParallel extension methods
  • WithExecutionMode and WithDegreeOfParallelism !!!
  • Due to the parallelism, the results will be unordered
    • AsOrdered can make you query run in parallel
    • But the result will be buffered and sorted synchronously
  • AsSequential can turn a parallel query into a sequential one back again
  • When exceptions are thrown, an AggregateException will be thrown with all exceptions
• ConcurrentCollections
  • Useful for multithreaded access environments
  • BlockingCollection
    • Thread-safe collection
    • Removing is blocking until some data is available
      • Calling the Take method will block the caller thread until any data is added to the collection
    • Adding is fast until certain limit
      • It uses ConcurrentQueue internally
    • Using CompleteAdding signals the collection that no more items will be added
      • This will unblock threads that were blocked by the Take method
      • Take has timeout? !!!
    • GetConsumingEnumerable returns a blocking enumerable
  • ConcurrentBag
    • A kind of thread-safe list
    • No particular order? !!!
    • Implements IEnumerable<T>, but creats a snapshot of the data when iteration starts
      • No new items will be iterated
  • ConcurrentStack and ConcurrentQueue
    • Thread-safe implementations of Stack and Queue
    • Also create data snapshot when iterating over data
  • ConcurrentDictionary
    • Also thread-safe
    • Mechanisms to add, update or get items based on previous value ("get the key X only if the value is Y")
      • Preventing inconsistent state/read

Objective summary

• A thread can be seen as a virtualized CPU.
• Using multiple threads can improve responsiveness and enables you to make use of multiple processors.
• The Thread class can be used if you want to create your own threads explicitly. Otherwise, you can use the ThreadPool to queue work and let the runtime handle things.
• A Task object encapsulates a job that needs to be executed. Tasks are the recommended way to create multithreaded code.
• The Parallel class can be used to run code in parallel.
• PLINQ is an extension to LINQ to run queries in parallel.
• The new async and await operators can be used to write asynchronous code more easily.
• Concurrent collections can be used to safely work with data in a multithreaded (concurrent access) environment.

Objective 1.2 - Manage multithreading

• Synchronization
  • The lock operator is translated to calls to Monitor.Enter and Monitor.Exit
    • How Monitor works? !!!
  • You should lock private reference types instances
    • Locking value types causes boxing
    • Then you loose the reference to the lock
  • Also avoid locking this and string
    • Locking this? !!!
    • Locking string can lock an object that is used in multiple places due to string-interning
      • string-interning is the proccess in which the compiler creates one object for several strings that have the same content
• volatile
  • Use the volatile keyword to avoid certain compiler optimizations
    • Optimizations like moving variable places, so instructions can be different
    • Not supported by all languages
• Interlocked class
  • Operations like n++ are not atomic
    • There is a need to read the value from the n variable, then add 1 to it
  • The Interlocked class helps making some operations atomic
    • Increment
    • Decrement
    • Exchange
    • CompareExchange (check the value before exchanging)
• Canceling tasks
  • Pass a CancellationToken to a Task
  • Then peroodically check the token to see if a cancellation was requested
    • IsCancellationRequested property
  • The token is created by an instance of CancellationTokenSource
  • To request a cancellation, call the Cancell method on the CancellationTokenSource instance
  • If you check the cancellation request (on a CancellationToken instance) and finishes your work, your Task will have a RanToCompletionState
    • But this does not mean that the Task has the work trully completed
    • To signal that the Task was cancelled, call ThrowIfCancellationRequested method
    • Then catch an AggregatedException
  • To cancel a Task based on a time, call Task.WaitAny passing a TimeSpan to the timeout parameter

Objective summary

• When accessing shared data in a multithreaded environment, you need to synchronize access to avoid errors or corrupted data.
• Use the lock statement on a private object to synchronize access to a piece of code.
• You can use the Interlocked class to execute simple atomic operations.
• You can cancel tasks by using the CancellationTokenSource class with a CancellationToken.

Objective 1.3 - Implement program flow

• Boolean expressions
  • && (and)
    • When all conditions are true
  • || (or)
    • When at least one contition is true
  • ^ (exclusive or)
    • When exactly one contition is true
• Control flow statements
  • if
  • while
  • do while
  • for
  • foreach
  • switch
  • break
  • continue
  • goto
  • Null-coalescing operator (??)
  • Conditional operator (?:)
• Iterating over collections
  • for
  • foreach
  • while
  • do while
• Delegates
  • A type that defines a method signature
  • Methods with the same signature are grouped into the delegate
    • Assigned (=) or added (+=)
    • All grouped methods are called throught the delegate
    • You also can remove method pointers (-=)
  • System.MulticastDelegate !!!
  • Covariance and contravariance !!!
• Lambdas
  • Anonymous functions
  • (x, y) => x + y can be read as x and y goes to adding x and y or x and y becames x adding and y
  • The return type and the params are infered by the compiler
  • Built-in delegate types (up to 16 parameters)
    • Func for return value functions
    • Action for void functions
  • Closures !!!
• event keyword
  • Wrappers around delegates to implement the publish-subscribe pattern with compiler checks
    • No null check needed
    • Compiler prevents unwanted access (like outside classes removing events)
    • Events cannot be directily assigned, only added
  • Events can be added outside of a class, but raised by the declaring class
  • Event pointers can be executed sequentially but is not guaranteeded
  • Exceptions stops the event execution flow
    • Control is done manually
    • Creating a try-catch
    • Obtaining the delegate list (EventHandler.GetInvocationList method)
    • Executing each delegate with Delegate.DynamicInvoke
    • Grouping the thrown exceptions
  • Event accessors add and remove can control how events are added or removed
    • Similar to the property accessors get and set

Objective summary

• Delegates are a type that defines a method signature and can contain a reference to a method.
• Delegates can be instantiated, passed around, and invoked.
• Lambda expressions, also known as anonymous methods, use the => operator and form a compact way of creating inline methods.
• Events are a layer of syntactic sugar on top of delegates to easily implement the publish-subscribe pattern.
• Events can be raised only from the declaring class. Users of events can only remove and add methods the invocation list.
• You can customize events by adding a custom event accessor and by directly using the underlying delegate type.

Objective 1.5 - Implement exception handling

• Handling exceptions
  • try-catch keyword
  • catch blocks should be specified as most-specific to last-specific
    • Generalized classes should be at the bottom
    • The runtime checks the exception types as they are specified
  • A try without a catch could be used to catch exceptions thrown from other languages that not inherit from System.Exception class
    • In C++ you can throw exceptions of any type
    • Useful on C# 1
    • Now exceptions are wrapped in a System.Runtime.CompilerServices.RuntimeWrappedException (inherits from System.Exception), so no more try without catch
  • finally blocks are excecuted after the catch blocks
    • When shutting down the application is safer than calling finally blocks, it's possible to use Environment.FailFast method that no finally block will be run
• Throwing exceptions
  • throw keyword
  • Create a instance of your desired Exception type
  • Rethrowing
    • When you catch an Exception is possible to do something (like log the exception) and then rethrow it
    • Only throw rethrows the original exception
    • Using throw with the exception variable rethrows it, but resets the call stack of the exception
    • When catching a certain exception type and then throwing another exception type, its recommended to store the older exception as an inner exception (inside the InnerException property)
    • It's possible to catch an exception and throw it on another thread
      • ExceptionDispatchInfo.Capture returns a ExceptionDispatchInfo
      • Then call the Throw method of the ExceptionDispatchInfo instance
      • The original exception stack trace will be preserved
  • Throwing exceptions for everything is not recommended
    • There is a performance hit because the runtime has to search all outer catch blocks (or a debuuger if no outer catch block is found)
• Custom exceptions
  • Inherit from System.Exception
  • The class should provide at least one parameterless constructor
  • It's also a best practice to add a few other constructors
    • One that takes a string, one that takes both a string and an Exception, and one for serialization
  • By convention, use the "Exception" suffix, add the Serializable attribute
  • Never inherit from System.ApplicationException
    • The original idea was that all custom exceptions should inherit from this class
    • But even the .NET Framework does not follow this convention, so this convention is meaningless

Objective summary

• In the .NET Framework, you should use exceptions to report errors instead of error codes.
• Exceptions are objects that contain data about the reason for the exception.
• You can use a try block with one or more catch blocks to handle different types of exceptions.
• You can use a finally block to specify code that should always run after, whether or not an exception occurred.
• You can use the throw keyword to raise an exception.
• You can define your own custom exceptions when you are sure that users of your code will handle it in a different way. Otherwise, you should use the standard .NET Framework exceptions.

Chapter summary

• Multithreading can help you create programs that are responsive and scalable. You can use the TPL, the Parallel class, and PLINQ for this. The new async/await keywords help you write asynchronous code.
• In a multithreaded environment, it’s important to manage synchronization of shared data. You can do this by using the lock statement.
• C# offers statements for making decisions—if, switch, conditional operator (?) and null- coalescing operator (??)—iterating (for, foreach, while, do-while), and jump statements (break, continue, goto, return and throw).
• Delegates are objects that point to a method and can be used to invoke the method. Lambda expressions are a shorthand syntax for creating anonymous methods inline.
• Events are a layer on top of delegates that help you with creating a publish-subscribe architecture.
• Exceptions are the preferred way to work with errors in the .NET Framework. You can throw exceptions, catch them, and run code in a finally block.

Chapter 2 - Create and use types

Objective 2.1 - Create types

• Three type categories
  • Value types
  • Reference types
  • Pointer types
• Enums
  • A special kind of value type
  • List of possible values
  • The first element has the value 0 and each element is icreased by 1
    • The starting index can be changed
  • Can be used as flags (with Flags attribute)
    • This will allow to use multiple combinations
• Value and reference types
  • A reference type contains a reference of the real value (a pointer to a memory address)
  • A value type contains the value directly
  • Heap and stack memory
    • Value types are stored in the stack memory
      • There are exceptions: boxed value types, fields of classes or lambdas
    • Reference types are stored in the heap memory, but its references are stored in the stack memory
  • Value types are small data structures that belongs together, like a coordinate (x and y positions)
    • The object is small
    • Is logically immutable
    • There are a lots of objects
  • Value types inherits from System.ValueType
    • It overrides some default functions
  • To create value types, use the struct keyword
    • Can have methods, fields, properties, contructors
    • Can not have:
      • Inheritance
      • Default empty constructor
• Named and optional arguments
  • With named arguments, you explicity specify which argument you are providing
  • Optional arguments have a default value if nothing is specified
• Adding some data
  • Fields can be marked as read-only
    • The field value can be set only during field declaration or object constructor
  • Consts is a compile time value
    • A field marked as const helps the compiler to make optimizations
  • Indexer is a way that enables object instances to be used as arrays
    • The index can be any type that you want (ref)
    • Indexers can be overloaded (ref)
  • A static field is a class field, not a instance field
    • This field can be accessed by the whole code
    • Its like a global variable
• Constructors
  • Explicitly declare the public default construct in classes if such a constructor is required
  • Ensure that your constructor takes as few parameters as possible
  • Map constructor parameters to properties in your class
  • Throw exceptions from instance constructors if appropriate
  • Do not call virtual members from an object inside its constructo
• SOLID
  • Single responsibility principle
    • A class should have only one responsability
  • Open/closed principle
    • An object should be open for extension but closed for modification
  • Liskov substitution principle
    • A base type should be replaceable by subtypes
  • Interface segregation principle
    • Client-specific interfaces over one general interface
  • Dependency Inversion principle
    • Depend upon abstractions, not concretions
• Generics
  • Avoids boxing/unboxing
  • Allow value types to be null (Nullable<T> or T?)
  • Constraints
    • Allows a generic type parameter to be constrained, like allowing only value types
    • where T: struct
  • default(T) returns the default value of the type
    • If T is a reference type, it will return null
• Extending types
  • Extension methods are a way to extend some types without modifying the original code
  • Can be used as a regular instance method
  • Need to be declared in a nongeneric, non-nested, static class
  • The method should be static
  • The first parameter should have the this keyword
    • This will indicate which type you are extending
    • Also gives access to the instance that will be used
    • static void SomeMethod(this string str)
• Overriding methods
  • Methods can be overridden when it is marked as virtual
  • Using sealed on an overridden method stops derived classes from overriding the method again

Objective summary

• Types in C# can be a value or a reference type.
• Generic types use a type parameter to make the code more flexible.
• Constructors, methods, properties, fields, and indexer properties can be used to create a type.
• Optional and named parameters can be used when creating and calling methods.
• Overloading methods enable a method to accept different parameters.
• Extension methods can be used to add new functionality to an existing type.
• Overriding enables you to redefine functionality from a base class in a derived class.

Objective 2.2 - Consume types

• Boxing and unboxing
  • Boxing is the process of storing a value type as a reference type on the heap memory
  • Unboxing is the process of taking the value type stored as a reference type on the heap memory and converting it to a value type
  • Unboxing is clear (explicit casts)
  • Boxing is not obvious
    • Passing a value type when a method expects an object
    • Calling GetType allways boxes a value type because this method is defined on the System.Object class and can not be overriden
    • When a value type is used as an interface (ex: IFormattable x = 3)
    • Using nongeneric collections with value types
• Conversions
  • Implicit conversions does not need special syntax
    • Happens when the compiler knows that it's safe to convert (ex: assign an int to a double because it fits)
    • Reference types to one of its base types or interfaces
  • Explicit conversion is called casting
    • Needs to be explicit because compiler does not allow unsafe value type conversions (int x = (double)1.2)
    • It also work with reference types by casting from a base type to a derived type
  • User-defined conversions
    • explicit and implicit operators
      • static implicit operator decimal(Money m)
      • static explicit operator int(Money m)
  • Implementing IFormattable allows the type to be used by the System.Convert class
  • System.BitConverter for noncompatible types !!!
  • System.Convert for compatible types !!!
  • Checking conversions
    • is operator allows to check if the type can be converted to another type
    • as operator try to convert the object to another type and returns null if the conversion is not possible
• Dynamic types !!!
  • DynamicObject is a base class that allows you to create dynamic types and controll the behaviour of getting or setting members, calling methods, etc
  • ExpandoObject enables you to get or set properties on a type, this avoid the creation of a type for every situation

Objective summary

• Boxing occurs when a value type is treated as a reference type.
• When converting between types, you can have an implicit or an explicit conversion.
• An explicit conversion is called casting and requires special syntax.
• You can create your own implicit and explicit user-defined conversions.
• The .NET Framework offers several helper methods for converting types.
• The dynamic keyword can be used to ease the static typing of C# and to improve interoperability with other languages.

Objective 2.3 - Enforce encapsulation

• Access modifiers
  • public has not access restriction
  • internal is limited to the current assembly
    • Expose them via InternalsVisibleTo assembly attribute
  • protected is limited to the containing class and derived classes
  • protected internal is limited to the current assembly or derived types
  • private is limited to the containing type
• Default accessibility
  • enum - public
  • class - private
  • interface - public
  • struct - private
• Properties
  • Seems like a regular field, but it has accessors: get and set
  • You can control how a property value is setted and read
• Explicit interface implementation
  • When an interface is implemented explicitly, the method/property can only be used by the interface
  • Useful when implementing multiple interfaces and method signatures conflicts

Objective summary

• Encapsulation is important in object-oriented software. It hides internal details and improves the usability of a type.
• Data can be encapsulated with a property.
• Properties can have both a get and a set accessor that can run additional code, commonly known as getters and setters.
• Types and type elements can have access modifiers to restrict accessibility.
• The access modifiers are public, internal, protected, protected, internal, and private.
• Explicit interface implementation can be used to hide information or to implement interfaces with duplicate member signatures.

Objective 2.4 - Create and implement a class hierarchy

• abstract classes can have implementation but no direct instantiation
• sealed classes are concrete classes but prevents inheritance
• Standard .NET interfaces
  • IComparable or IComparable<T>
    • Used to sort elements
    • Implements the CompareTo(object) method
    • This method expects -1 when the current instance preceeds the specified object, 0 when they are equal and 1 when the current instance follows the specified object
  • IEnumerable or IEnumerable<T>
    • Implement the iterator pattern
    • Can be used with foreach calls
    • yield keyword made possible to easily implement enumerables
  • IDisposable
    • Used to free unmanaged resources
    • Can be used with a try-finally block (manually call Dispose()) or with the using keyword
  • IUnknown !!! (ref)
    • Used to create COM Interop classes
    • When the compiler can't generate the wrapper classes

Objective summary

• Inheritance is the process in which a class is derived from another class or from an interface.
• An interface specifies the public elements that a type must implement.
• A class can implement multiple interfaces.
• A base class can mark methods as virtual; a derived class can then override those methods to add or replace behavior.
• A class can be marked as abstract so it can’t be instantiated and can function only as a base class.
• A class can be marked as sealed so it can’t be inherited.
• The .NET Framework offers default interfaces such as IComparable, IEnumerable, IDisposable and IUnknown.

Objective 2.5 - Find, execute, and create types at runtime by using reflection

• Attributes
  • A kind of metadata
  • Can be used anywhere and the usage can be controlled via AttributeUsage attribute
  • Attribute.IsDefined(Type, Type) static method can be used to check if some type is annotated with a certain attribute type
• Reflection
  • Slower than static code (reflection is executed at runtime)
  • Query information about types
  • Enumerate types inside an assembly
  • Iterate over class fields (even private ones)
  • Invoke methods
• CodeDOM
  • Can generate a source file or a binary assembly that can be executed
• Expression trees
  • Represent code
  • Compiled to Action or Func<T>
  • Which is better, CodeDOM or Expression trees? !!!
    • Do they differ in the use case?

Objective summary

• A C# assembly stores both code and metadata.
• Attributes are a type of metadata that can be applied in code and queried at runtime.
• Reflection is the process of inspecting the metadata of a C# application.
• Through reflection you can create types, call methods, read properties, and so forth.
• The CodeDOM can be used to create a compilation unit at runtime. It can be compiled or converted to a source file.
• Expression trees describe a piece of code. They can be translated to something else (for example, SQL) or they can be compiled and executed.

Objective 2.6 - Manage the object life cycle

• Garbage collection
  • The process of cleaning up the memory from unused objects
  • It uses the mark and compact algorithm
    • Marks the object that are still being referenced
    • Compact the memory by cleaning up the objects that are not marked as being referenced
  • All threads are frozen when the GC kicks in
    • But it starts when there is no memory for a new allocation
    • Or when the SO signals that it is low on memory
• Finalizer vs IDisposable
  • A finalizer is called by the GC, so, finalizers are unpredictable
  • Dispose is called by code, so, it is predictable
  • Good to use System.GC.SuppressFinalize(this) when disposing, because it removes the object from the GC's finalization list
• WeakReference !!!
  • Does not hold a real reference to an item on the heap memory
  • Can be garbage collected

Objective summary

• Memory in C# consists of both the stack and the heap.
• The heap is managed by the garbage collector.
• The garbage collector frees any memory that is not referenced any more.
• A finalizer is a special piece of code that’s run by the garbage collector when it removes an object.
• IDisposable can be implemented to free any unmanaged resources in a deterministic way.
• Objects implementing IDisposable can be used with a using statement to make sure they are always freed.
• A WeakReference can be used to maintain a reference to items that can be garbage collected when necessary.

Objective 2.7 - Manipulate strings

• Strings
  • Immutable
  • Is a reference type but == and != are overloaded to compare strings by value
  • When there is two equal literal strings, the compiler ensures that only one string is created into the memory
    • Called string interning
• StringBuilder
  • Avoid to many allocations
    • Concatenating strings with + generates many allocations
    • "a" + "b" + "c"
      • Generates an allocation for the "ab" string and then one more for the "abc" string
  • Useful inside loops
  • String literals can be optimized by the compiler
• StringWriter and StringReader
  • Both use StringBuilder internally
  • Both are an adaptation of the StringBuilder to be used as TextWriter and TextReader
• String search
  • IndexOf, LastIndexOf, StartsWith, EndsWith and Substring are the most common methods
  • String methods can be culture sensitive
    • Avoid a method that do not use StringComparison flag !!!
• String implements the IEnumerable<char> interface
• Formatting
  • The process of converting an instance type to a string representation
  • CultureInfo can provide information on how to format some data
  • It is good to save data in a culture-sensitive way
    • Because you use CultureInfo to display data
• IFormatProvider !!!
  • Returns specific information for formatting a type
• IFormattable
  • Allow your type to be used by the System.Convert class
  • Also support composite formatting !!!
    • string.Format

Objective summary

• A string is an immutable reference type.
• When doing a lot of string manipulations, you should use a StringBuilder.
• The String class offers a lot of methods for dealing with strings like IndexOf, LastIndexOf, StartsWith, EndsWith, and Substring.
• Strings can be enumerated as a collection of characters.
• Formatting is the process of displaying an object as a string.
• You can use format strings to change how an object is converted to a string.
• You can implement formatting for your own types.

Chapter summary

• C# uses types such as class, struct, interface, and enum. Types have can have members such as methods, events, fields, properties, indexed properties, and constructors.
• When working with types, you sometimes need to convert between them. This can be done either implicitly or explicitly. When creating your own types, you can add support for these types of conversions.
• You use accessors such as public, private, protected, internal, and protected internal to enforce encapsulation and accessibility. Properties can be used to encapsulate data.
• An object hierarchy can be created by inheritance, and you can have both interface and class inheritance. By marking members as virtual, a derived class can override the member.
• Reflection is the process of inspecting metadata at runtime. Attributes can be used to add metadata to a type.
• C# is a managed language, which means that a garbage collector makes sure that all managed objects are freed from memory whenever they are no longer in use.
• Strings can be used for text. The string type is immutable. The .NET Framework offers a StringBuilder for manipulating large amounts of text. Strings can be searched, enumerated, and formatted for display.

Chapter 3 - Debug applications and implement security

Objective 3.1 - Validate application input

• Data integrity
  • Entity
    • Eacho entity should be uniquely identifiable (like primary keys in database)
  • Domain
    • Data types and possible allowed types
  • Referential
    • Relationship between entities
  • User-defined
    • Business rules
• User-defined integrity can be achieved with attributes on entity properties
  • Required, MaxLength, Range and RegularExpr are .NET defined attributes
  • ValidationContext class !!!
• Parse and TryParse
  • Input usually is a string, so there is a need to parse strings to specific types
  • Parse should be used when you are certain that the conversion will succeed
    • This method throws exceptions
  • TryParse let you handle invalid data gracefully
    • It returns a boolean indicating that the conversion was cuccessfull
  • decimal and DateTime have format styles and CultureInfo to do its job
• Convert
  • Is used to convert between base types
• Regular expressions
  • System.Text.RegularExpressions
  • RegExpr.Match and RegExpr.Replace
• Validating XML and JSON
  • Using JavascriptSerializer and help converting and validating JSON
  • Validating XML
    • XML is described by an XSD (XML Schema Definition)
    • xsd.exe generates XSD files from XML files
      • The XSD should be refined manually later
    • Load an XmlDocument and add a schema to it (.Schemas.Add)
      • Then call Validate passing a ValidationEventHandler delegate
      • The delegate will be called when errors or warnings occur

Objective summary

• Validating application input is important to protect your application against both mistakes and attacks.
• Data integrity should be managed both by your application and your data store.
• The Parse, TryParse, and Convert functions can be used to convert between types.
• Regular expressions, or regex, can be used to match input against a specified pattern or replace specified characters with other values.
• When receiving JSON and XML files, it’s important to validate them using the built-in types, such as with JavaScriptSerializer and XML Schemas.

Objective 3.2 - Perform symmetric and asymmetric encryption

• Symmetric and asymmetric encryption
  • Since algorithms are public, we should keep our key safe
    • Encryption algorithms uses keys to control the process
  • Symmetric uses one key
  • Asymmetric uses two keys
    • One key is private and the other is public
  • IV (Initialization Vector)
    • Is used to generate "random" outputs, making difficult to decrypt
  • Symmetric encryption can be done with AesManaged class (Advanced Encryption Standard)
    • Inherits from SymmetricAlgorithm class
    • CreateEncryptor and CreateDecryptor methods returns a ICryptoTransform
    • CryptoStream is used to read or write data into a Stream using the provided ICryptoTransform
  • Asymmetric encryption is done with the RSACryptoServiceProvider and DSACryptoServiceProvider
  • Keys can be stored into a key container
    • Provide the key container name to the CryptoServiceProvider via CspParameters (KeyContainerName property)
• Hash
  • The proccess of mapping large datasets to small (and fixed) datasets
  • Complex objects can be described by an integer
  • Override GetHashCode
  • SHA256Managed can generate hash codes efficiently
• Certificates
  • Part of the PKI (Public Key Infrastructure)
  • X.509 is a widely used stantard for digital certificates
    • makecert.exe helps create certificates for testing
    • X509Store can read the installed certificates of a specific custom certificate store
      • It returns instances of X509Certificate that you can access the public and private keys (instances of RSACryptoServiceProvider)
• CAP - Code Access Permissions !!!
  • Also known as CAS (Code Access Security)
  • Apps have full trust
  • Every CAP has
    • The right to access a protected resource
    • The right to perform a protected operation
  • The whole call stack is verified for permissions
• Securing string data
  • Strings can be moved by the GC and leave immutable copies around the memory
  • When in low memory, a string could be written to a page file on disk in plain text
    • The same could happen when the memory ins dumped
  • Impossible to force the GC to remove all copies of the string
  • System.Security.SecureString
    • Encrypts its value
    • Pinned to a specific memory location (not moved by GC)
    • IDisposable
    • One char at time (no original string)
    • Use System.Runtime.InteropServices.Marshal to find the secure string IntPtr and then its value

Objective summary

• A symmetric algorithm uses the same key to encrypt and decrypt data.
• An asymmetric algorithm uses a public and private key that are mathematically linked.
• Hashing is the process of converting a large amount of data to a smaller hash code.
• Digital certificates can be used to verify the authenticity of an author.
• CAS are used to restrict the resources and operations an application can access and execute.
• System.Security.SecureString can be used to keep sensitive string data in memory.

Objective 3.3 - Manage assemblies

• Assemblies
  • Self-contained DLLS (with assembly's manifest inside)
  • Language-neutral (can be used by languages like F# and VB)
  • Can be versioned (multiple versions without conflict)
  • Strong-named assemblies
    • Signed with your private key
    • sn.exe tool (generates .snk files)
    • al.exe tool (generates assembly manifests)
    • Visual Studio can sign assemblies automatically
    • Public token is a hash of the public key taken the last eight bytes
    • Authenticode !!!
  • Global Assembly Cache (GAC)
    • Machine-wide assembly storage
    • Use GAC when an assembly is shared across multiple apps, or for enchanced security (only adm can change the GAC), or to deploy multiple versions of the same assembly
    • gacutil.exe lists, install and uninstall assemblies
  • Assembly versions
    • Major.Minor.Build.Revision
    • Major for many new features
    • Minor for small changes
    • Build is generated by the build server
    • Revision is for production patches
    • AssemblyFileVersionAttribute/AssembleVersionAttribute !!!
    • Side-by-side hosting
      • Multiple versions of an assembly are hosted on one computer
    • Assembly binding
      • Influenced by:
        • App config files
        • Publisher policy files !!!
        • Machine config files
      • Publisher policy files can redirect the assembly bindings
      • Extra locations can also be specified
      • Codebase element specify a file on the network or on the web

Objective summary

• An assembly is a compiled unit of code that contains metadata.
• An assembly can be strongly signed to make sure that no one can tamper with the content.
• Signed assemblies can be put in the GAC.
• An assembly can be versioned, and applications will use the assembly version they were developed with. It’s possible to use configuration files to change these bindings.
• A WinMD assembly is a special type of assembly that is used by WinRT to map nonnative languages to the native WinRT components.

Objective 3.4 - Debug an application

• Build configurations
  • Debug mode
    • Adds extra informatiion and operations for debugging purposes
    • No compiler optimizations
  • Release mode
    • Fully optimized
    • No debug information or operations are added
• Compiler directives
  • #IF
    • Can be used to test directives
    • Has #else and #endif
    • Can have the same operators as the C# language
  • Define new symbols using the #DEFINE directive
    • It should appear before any other code in the file
  • Directives should be avoided if possible
    • A scenario that is ok to use is when targeting multiple platforms (WinRT can have a slightly different api, for example)
  • #undef can be used to remove the definition of a symbol
  • #warning an #error will trigger a compiler warning and error respectivelly
  • Code generation features
    • Its possible to map the lines back to the original file for debug
    • #line {number} '{file name}'
      • The file name is optional
  • #pragma is used to disable warnings
  • Compiler directives are useful, but can be a pain to wrap calls to methods every time
    • ContitionalAttribute
    • Pass the directive as paramater ([Conditional("Directive")])
    • The compiler removes calls that does not match the directive
  • Overriding ToString helps create nice visualizations of objects, but what to do when this visualization is only to debug?
    • DebugDisplayAttribute
    • Is not shown in release build mode
• PDB Files (Program Database)
  • Contains extra data source that is useful for debug
  • /debug:full
    • PDB is generated and the assembly is modified to have debug information
  • /debug:pdbonly
    • Only the PDB file is generated and no assembly is modified
    • Recommended for release build
  • Both asembly and PDB have the same GUID
  • Symbol Server exposes PDB files
    • TFS (Team Foundation Server) helps create an internal symbol server
  • PDB Files contains two collections of information: private and public data
    • Public data contains items that are accessible from one source to another
    • Private data contains data that is visible in only the object, such as local variables
  • pdbcopy tool can remove private data so you can share the PDB file publicly

Objective 3.5 - Implement diagnostics in an application

• Logging and tracing
  • Tracing is used to monitor the app execution
  • Logging is used for error reporting
  • Logging can be achieved by the Debug class which outputs to the Output Window
  • TraceSource is used for tracing
    • Methods can receive an TraceEventType
      • Critical for serious and irrecoverable errors
      • Error for problems that has been handled or recovered from
      • Warning for something unusual
      • Information indicates that the process is running correctly, but there is information that is interesting
      • Verbose for anything that is not the above
      • Start, Stop, Suspend, Resume, Transfer for logical flow
  • ID number is used to group certain events
  • Listeners
    • Both Debug and TraceSource has a Listeners property
    • By default, a DefaultTraceListener is used
    • Few useful listeners
      • ConsoleTraceListener
      • DelimitedTraceListener
      • EventLog
      • EventSchema
      • TextWriter
      • XmlWriter
    • Listeners can be defined via configuration files
      • Filters and switches can be used
      • Switches are for everything
      • Filters are per listener
    • New listeners can be created by inheriting from the TraceListener class
    • Shared listeners can be used by multiple trace sources in app.config
    • Events can be written to the Windows Event Log using the EventLog class
      • The running account should have the appropriate permission
      • new EventLog().WriteEntry("");
    • Its possible to view messages using the Windows Event Viewer or programmatically using the Entries property of the EventLog class
    • EventLog class has an EntryWritten event to subscribe to new events
• Profiling
  • The basic measurement way is using the StopWatch class
  • Visual Studio 2012 includes a set of profiling tools (only for Ultimate, Premium or Professional versions)
  • The Performance Wizzard can help
    • CPU Sampling
      • The most lightweight option
    • Instrumentation
      • Injects code into the compiled file that captures timing information
    • .NET Memory Allocation
      • Monitor allocations and garbage collections
    • Resource contention data
      • Analyses waits for shared resources
      • Useful for multithreaded applications
  • Performance counters
    • perfmon.exe shows some Windows Performance Counters about hardware, but it's possible to create a bunch of custom perf. counters
    • PerformanceCounter class can be used to read and create perf. counters:
      • NumberOfItems
      • RateOfCountsPerSecond
      • AverageTimer

Objective summary

• Logging and tracing are important to monitor an application that is in production and should be implemented right from the start.
• You can use the Debug and TraceSource classes to log and trace messages. By configuring different listeners, you can configure your application to know which data to send where.
• When you are experiencing performance problems, you can profile your application to find the root cause and fix it.
• Performance counters can be used to constantly monitor the health of your applications.

Chapter summary

• Validating application input is important to ensure the stability and security of your application. You can use the Parse, TryParse, and Convert functions to parse user input. Regular Expressions can be used for matching patterns.
• Cryptography uses symmetric and asymmetric algorithms together with hashing to secure data.
• Code access permissions can be used to restrict the types of operations a program may execute.
• An assembly is a self-contained unit that contains application code and metadata. An assembly can be signed, versioned, and shared by putting it in the GAC.
• By selecting the correct build configurations, you can output additional information to create program database files that can be used to debug an application.
• By using logging, tracing, and performance counters, you can monitor an application while it’s in production.

Chapter 4 - Implement data access

Objective 4.1 - Perform I/O operations

• Working with files
  • DriverInfo class can access drivers information
  • DirectoryInfo/Directory
    • DirectoryInfo class represents a directory and is recommended when you are performing multiple operations on the same directory
    • Static Directory class has the same features as the DirectoryInfo class, but is recommended for single operations on a directory
  • DirectorySecurity contains security information about a directory ant it's possible to add new security rules !!!
  • Subdirectories can be listed using a search pattern (ex *.txt) and a SearchOption (All or RootOnly)
    • EnumerateDirectories can be more efficient because it's possible to start enumerating the collection before it's been completely retrieved
  • FileInfo/File
    • The same as DirectoryInfo and Directory static class
  • The Path class is helpful when we are manipulating paths
    • It's possible to combine paths, extract extensions, filenames, directory names and also root paths
    • For temporary files, it's possible to get the temporary path, generate a random filename or directory name; and create a temporary file
  • The Stream class is an abstration of a sequence of bytes
    • It provides a generic interface for all types of IO
    • Reading is when you get series of bytes
    • Writing is when you are sending a series of bytes
    • Seeking is when available, you change the stream cursor for reading or writing
    • The FileStream is a class that deals with streams of files
    • Encoding and decoding
      • The process of converting chars into bytes and vice-versa
    • It's possible to combine streams to create complex operations, like using the GZipStream, that receives the final stream as parameter. The same is valid to the BufferedStream
• Comunicating over the network
  • WebRequest and WebResponse are abstract classes that deals with web communications
  • WebRequest.Create returns a concrete implementation depending on the protocol
• Async IO operations
  • Sometimes, something called thread pool starvation happens. This is when severals threads are waiting for the IO and new threads are queued. All of this happen with CPU load of 0%
  • When async/await keywords where introduces in C# 5, this king of problem can be avoided
  • Many synchronous IO methods have an async equivalent like the Write method of thie FileStream class that has an WriteAsync equivalent
  • Async methods returns a Task or Task<T> when there is an useful return value
  • Real async IO operation on the FileStream class is done by passing true for the useAsync parameter
  • When using multiples await, each start when the previous is finished
    • To parallelize, it's possible to store the Tasks and call Task.WhenAll

Objective summary

• You can work with drives by using Drive and DriveInfo.
• For folders, you can use Directory and DirectoryInfo.
• File and FileInfo offer methods to work with files.
• The static Path class can help you in creating and parsing file paths.
• Streams are an abstract way of working with a series of bytes.
• There are many Stream implementations for dealing with files, network operations, and any other types of I/O.
• Remember that the file system can be accessed and changed by multiple users at the same time. You need to keep this in mind when creating reliable applications.
• When performing network requests, you can use the WebRequest and WebResponse classes from the System.Net namespace.
• Asynchronous I/O can help you create a better user experience and a more scalable application.

Objective 4.2 - Consume data

• Working with a database
  • Two types of data: connected and disconnected
  • Connected data
    • Explicit connection with a database
    • Uses SQL that ADO.NET forward to the database
    • Connection, Command and DataReader objects
  • Disconnected data
    • DataSet and DataTable classes that mimic a memory database
    • Changes are synche using DataAdapter class
  • Providers
    • The .NET data provider is a thin layer that is used to connect to a database, execute commands an work on the resulting data
  • Connecting
    • Database connections inherit from DbConnection that holds an unmanaged database connection and implements IDisposable because of this
    • Connection strings can be easily created using one of the many child of the DbConnectionStringBuilder class
      • Connection strings can also be stored in the app.config/web.config files inside the connectionStrings section
      • Those stored strings are accessed via ConfigurationManager.ConnectionStrings property
      • Connection strings for different configurations !!!
      • ADO.NET uses connection pools to improve performance
  • Reading data
    • Create a DbConnection instance
    • Create a DbCommand instance with the SQL statement
    • Open the database connection
    • Get a DataReader using DbCommand's ExecuteReader method (there is an async option)
    • Call DataReader's Read method until it returns false
      • This will read a line from the resulting data
    • Access the columns by index or column name
    • DataReader is a forward-only stream of rows
      • It also has methods that map the value of a column to the CLR corresponding type (like GetInt32, GetString, GetDecimal, etc)
      • It has de concept of result sets where you group multiple SQL statements and query them once
  • Updating data
    • Data can be updated using DbCommand's ExecuteNonQuery method
    • The method receives an DML statement and return the number of rows affected by the last statement
  • Parameters
    • Parametrized commands are useful to avoid SQL attacks
    • DbCommand.Parameters.AddWithValue
      • The parameters should be prefixed with @
    • This can improve performance because the database will receive a more generic query and can precompile it !!!
  • Transactions
    • Its recommended to use transactions when you have multiple queries that should be grouped together
    • It's all win or all fail
    • TransactionScope class simplify the use of transactions
      • Call the Complete method to finish the transaction
    • When you have nested connections, databases, resources types inside your transaction, the transaction is promoted to a distributed transaction !!!
      • Then not needed, it impacts performance
  • ORM
    • Useful to speedup the development, but its not that performatic
    • Entity Framework is a Microsoft framework that maps your database to your objects
• Webservices
  • It's another option for storing data
  • You can exchange data between apps in a loosely coupled way
  • The Microsoft solution for webservices is called Windows Communication Foundation (WCF)
  • Create a contract
    • Annotate the class with ServiceContract
    • Annotate the method with OperationContract
  • An .svc file is generated
    • It contains instructions for how to host the service in IIS
  • Visual Studio creates a proxy class for previously created WCF (as service references)
    • This proxy class uses the configuration file for the bindings and addrress of the service
• Consuming XML
  • XmlReader is fast and forward-only
  • XmlWriter is fast and forward-only
  • XmlDocument is an in-memory representation of the XML and supports navigation and editing
  • XPathNavigator is used for better navigation
• Consuming JSON
  • Newtonsoft.Json !!!

Objective summary

• ADO.NET uses a provider model that enables you to connect to different types of databases.
• You use a DbConnection object to create a connection to a database.
• You can execute queries that create, update, read, and delete (CRUD) data from a database.
• When creating queries it’s important to use parameterized queries so you avoid SQL injection.

Objective 4.3 - Query and manipulate data and objects by using LINQ

• C# features that made LINQ possible
  • Implicit typed variables
  • New object initialization syntax
  • Lambda expressions
  • Extension methods
  • Anonymous types
• Using LINQ
  • Two types of syntax
    • Query syntax (from in where select)
    • Method syntax (.Where, .Select, etc)
  • Query syntax is translated to method calls by the compiler
  • LINQ operators are extension methods on IEnumerable<T>
  • Standard query operators
    • Where to filter elements
    • OrderBy for sorting
    • Multiple sources are possible with query syntax
    • Projection to map a row to a type
    • Group to group the data using a certain property or object as a key
    • Join to combine from many sources but specify a rule for comparison
    • Paging
      • Skip to skipe a number of elements
      • Take to limit how much elements are used
• How LINQ works
  • Internally, LINQ uses the yield operator (iterator pattern)
  • Because of the iterator pattern, the LINQ queries are not executed until the result is iterated (called deferred execution)

Objective summary

• LINQ, which stands for Language Integrated Query, is a uniform way of writing queries against multiple data sources.
• Important language features when working with LINQ queries are implicit typing, object initialization syntax, lambdas, extension methods, and anonymous types.
• You can use LINQ with a method-based syntax and the query syntax.
• LINQ queries are deferred-execution, which means that the query executes when it is first iterated.
• You can use LINQ to XML to query, create, and update XML.

Objective 4.4 - Serialize and deserialize data

• Serialization/deserialization is the process of transform text/binary data into language objects an vice-versa
• It's a complex process because objects can easily have cyclic references
• Optimization can be done using a Data Transfer Object (DTO)
• Default mechanisms
  • XmlSerializer
  • DataContractSerializer
  • BinaryFormatter
• XmlSerializer
  • Does not have the highest performance
  • Work only with public fields
  • Mark the classes with Serializable attribute so the object graph is checked (can throw Exception)
  • The process can be controlled using some attributes
    • XmlIgnore
    • XmlAttribute
    • XmlElement
    • XmlArray
    • XmlArrayItem
• BinaryFormatter
  • Smaller result than XML
  • Also need classes marked as Serializable
  • Private fields are serialized
  • Strictier than XML serialization
    • Not found fields throw exceptions
    • OptionalField attribute indicates that the field was added later
  • Four phases that allow the control of the process
    • Use attributes on methods
      • OnDeserialized
      • OnDeserializing
      • OnSerialized
      • `OnSerializing``
    • All of the methods receives a StreamingContext as a parameter
  • When serializing sensitive data, is recommended to implement ISerializable interface
    • This allow to control which field and how it will be serialized/deserialized
• DataContractSerializer
  • Outputs a XML
  • Use DataContract attribute instead of Serializable
  • Members are not serialized by default
    • It's necessary to annotate members with DataMember
• DataContractJsonSerializer
  • The same as DataContractSerializer but outputs a JSON

Objective summary

• Serialization is the process of transforming an object to a flat file or a series of bytes.
• Deserialization takes a series of bytes or a flat file and transforms it into an object.
• XML serialization can be done by using the XmlSerializer.
• You can use special attributes to configure the XmlSerializer.
• Binary serialization can be done by using the BinaryFormatter class.
• WCF uses another type of serialization that is performed by the DataContractSerializer.
• JSON is a compact text format that can be created by using the DataContractJsonSerializer.

Objective 4.5 - Store data in and retrieve data from collections

• Arrays
  • Useful with fixed number of items
  • It's required to specify the number of items
  • Zero-based (the first elements starts at the 0 index)
  • It's possible to only specify the items, then the compiler will check the number of items
  • Arrays are reference types
  • Jagged arrays are arrays of arrays
• Generic vs nongeneric collections
  • When using one specific type (or base type), it's better to use a generic collection
    • No casting will be required
  • If a value type is used as a parameter, there are scenarios which boxing can occur
    • When the types does not implement IEquatable<T> or IComparable<T>
• List
  • Expandable (no need to specify the size)
  • Add, remove, access by index, search and sort is possible
• Dictionary
  • Key-value pairs
  • Retrieved by key
  • Implemented as hash table
    • Very fast, close to O(1)
  • Work with the KeyValuePair<TKey, TValue> structure
• Sets
  • HashSet class
  • Mathematicals operations
• Queues and Stacks
  • Queue is First In, First Out (FIFO)
  • Stack is Last In, First Out (LIFO)

Objective summary

• The .NET Framework offers both generic and nongeneric collections. When possible, you should use the generic version.
• Array is the most basic type to store a number of items. It has a fixed size.
• List is a collection that can grow when needed. It’s the most-used collection.
• Dictionary stores and accesses items using key/value pairs.
• HashSet stores unique items and offers set operations that can be used on them.
• A Queue is a first-in, first-out (FIFO) collection.
• A Stack is a first-in, last-out (FILO) collection.
• You can create a custom collection by inheriting from a collection class or inheriting from one of the collection interfaces.

Chapter summary

• You can use classes such as Drive, DriveInfo, Directory, DirectoryInfo, File, and FileInfo to work with the file system. All I/O uses Streams, which are an abstraction over a series of bytes.
• Asynchronous code is important for long-running operations to improve responsive- ness and scalability.
• When working with a database, you can use ADO.NET to establish a connection, execute commands, and retrieve results.
• The .NET Framework has support for working with XML by using classes such as XmlWriter, XmlReader, and XmlDocument, or by using LINQ to XML.
• LINQ offers a uniform way of querying different data sources.
• Serializing and deserializing is the process of transforming an object to a flat file or a series of bytes, and vice versa.
• The .NET Framework offers a comprehensive set of collections that you can use in different scenarios.