Monday, June 9, 2008

Hw vs Sw

Men and women, hawks and doves ...
Hw and Sw are like the ying and yang of the information revolution. Of course, all dualities hide a continuum and literally one cannot live without the other.
People think they understand hw (maybe they do), but I'm pretty sure they do not understand sw (I doubt I do either).
This entry tries to enumerate differences and synergies among both.


Sw predates Hw (in the sense of sw-running hw). Seems perplexing but it is true. It took some time to realize the minimal hw needed for sw to run on it. It is great that (finite) Turing-like hw is enough to run any computable sw.


Hw employs a minimal fragment of the people sw employs.
Hw produces a millionth of the different products sw produces.
Hw evolution has been consistently macro-predicted, the latest models of sw release take fixed points in time with no sure prediction of achieved functionality (but we get some).
Hw mistakes can destroy manufacturers. Sw mistakes have to be labeled as features.
A mistake in a hw piece requires its replacement. A mistake in a sw piece requires a patch write.
We have 2 centuries of experience with mass-produced goods, but only a few decades of sw quality evaluation.
A faulty Hw piece is easy to identify. A faulty Sw piece can take millions of people years to identify.
It is extremely expensive to produce 1 new piece of hw, a mass-market is needed. It is relatively inexpensive to produce a non-complex piece of sw, but it is extremely expensive to grow from it.
Hw duplication is expensive. Sw duplication is almost free.
Hw is Sw that has solidified too early. Sw is Hw that has not been fully debugged yet.
There is not such thing as a bug-free (non-trivial) sw.
Hw platforms change slowly. New sw platforms are developed every few years. (Understand platform as ecosystem, integration ...)
MS is a closed sw platform builder that owns an open hw platform. Apple is a closed hw platform builder that owns a (semi-)open sw platform. Linux is an open sw infrastructure to run sw on multiple hw platforms. (see open/closed, here open can be semi-open)
You can build Hw by componentization, duplication ... You can try to build sw by piling up stuff, but you usually just get a big mess.
Current Hw is no longer execution-predictable. Current sw no longer fits into one's head.
You cannot easily start again with a Hw architecture. You'd do better redesigning your sw architecture (and emulating the old one if needed).
The barriers to entry into Hw are colossal. A monkey with a PC could create good sw.
You can educate a Hw engineer predictably. Education alone cannot produce good Sw engineers.
A Hw engineer can plan his career. A Sw engineer considers himself lucky if he can plan his vacation.
When hw is badly done, only publicity can save you. When sw is badly done, you can get away with murder.
Good Hw guys usually were good mechanical tinkering children. Good Sw guys usually were insufferable philosophical kids.
Hw is provided as is but has full warranty. Sw is tailored to death, but has no warranty whatsoever.
Hw guys are to surgeons as Sw guys are to psychologists.

...

Saturday, June 7, 2008

Design Patterns

No blog about design can avoid mentioning GoF's book. It took the world by storm in the middle 90s and little by little it percolated to mainstream programming. After it there are JEE patterns, distributed patterns, enterprise patterns ... you name it.
Gof's comes from the Smalltalk tradition and shows how to add flexibility to class-based OO languages like C++ and Java. Gof's defines a common vocabulary and typical design solutions for some problems (based on the architectural work of Alexander). In this century critics have reacted to this, despising (with reason) those who use patterns like recipes (see Kerivesky's work for a reasonable alternative) and languages who need these patterns as "broken" (partly true). Newer languages based on the class-based OO tradition have tried to simplify or directly implement as idioms these patterns. Gof's is a book EVERY designer should read and understand, anyway.

I'll enumerate (with some notes) all Gof patterns (and some more here). I'll try to show some Scala implementation of these patterns as a personal exercise. UML 2 tries to simplify pattern representation (aiding to represent other high-level multiclass 'packages'). There is a nice (if slighty mocking) pseudo elements table relating them (worth printing to despise unknowing 'hackers').


Creational
These are patterns that try to simplify and generalize object creation strategies.
Singleton; Factory Method; Abstract Factory; Builder; Prototype

Structural
These help building common object structures.
Adapter; Decorator; Façade; Bridge; Composite; Flyweight; Proxy

Behavioral Patterns
These help building common collaboration patterns.
Chain of Responsibility; Visitor; Observer; Iterator; Mediator;State; Command; Mememto; Strategy

And, finally Gof includes a pseudojoke to help remember that no 'static' language is sacred and that scripting is usually needed: Interpreter.

Great book, (not the last word) but a common ground for OO developers.



Also check: JEE patterns, enterprise patterns, analysis patterns, ...

Davis' 201 principles of software development

This is a rewriting of Alan Davis' excellent 201 principles of sw dev list of principles. Some could be in several places (but I imposed myself to only show them once ...).
Some abilities can only be acquired by doing; Others will not be acquired early enough (if ever) unless good formal education is received early. Even others cannot be acquired (or only with too much effort) by some people.

Not all are valid now and you may not agree with all of them.

Outdated
CASE Tools Are Expensive (economically)
Be Optimistic About Hardware Evolution (current hw evolution requires mastery of concurrency).


Quality
Quality Is #1; Quality Is in the Eyes of the Beholder; Don't Try to Retrofit Quality; Productivity and Quality Are Inseparable; High-Quality Software Is Possible; Poor Reliability Is Worse Than Poor Efficiency
Comments
Avoid Broken windows; Avoid it works 'somehow'

People
Give Products to Customers Early; Communicate with Customers/Users; Align Incentives for Developer and Customer; Align Reputation With Organization; Follow the Lemmings With Care; Give Software Tools to Good Engineers;
Comments
Only those Doing can tell how, others can tell what.

Building (and Prototyping)
Grow Systems Incrementally; Change During Development Is Inevitable; If Possible, Buy Instead of Build; Build Software So That It Needs a Short Users' Manual; Every Complex Problem Has a Solution; Different Languages for Different Phases; Technique Before Tools; Use Tools, but Be Realistic; CASE Tools Are Expensive (in the process sense); "Know-When" Is as Important as Know-How; Stop When You Achieve Your Goal; Plan to Throw One Away; Build the Right Kind of Prototype; Build the Right Features into a Prototype; Build Throwaway Prototypes Quickly; The More Seen, the More Needed

Comments
If you PLAN to throw one away, you WILL throw two!



Documentation (and communication)
Record Your Assumptions; Use Documentation Standards; Every Document Needs a Glossary; Every Software Document Needs an Index; Use the Same Name for the Same Concept;
Comments
If you plan to document late you won't, or worse; Unsynch'ed docs are worse than no docs (except for saving face);
If you cannot do docs you are NOT a good developer: CANNOT deal with multiple languages, NO abstraction capability; NO design; CANNOT communicate


Education
Know Formal Methods; Don't Ignore Technology; Research-Then-Transfer Doesn't Work; Take Responsibility
Comments
If you are not educated, you will AT BEST reinvent all wheels


Requirements
Poor Requirements Yield Poor Cost Estimates; Determine the Problem Before Writing Requirements; Determine the Requirements Now; Fix Requirements Specification Errors Now; Prototypes Reduce Risk in Selecting User Interfaces; Record Why Requirements Were Included; Identify Subsets; Review the Requirements; Avoid Design in Requirements; Use the Right Techniques; Use Multiple Views of Requirements; Organize Requirements Sensibly; Prioritize Requirements; Write Concisely; Separately Number Every Requirement; Reduce Ambiguity in Requirements; Augment, Never Replace, Natural Language ; Write Natural Language Before a More Formal Model; Keep the Requirements Specification Readable; Specify Reliability Specifically; Specify When Environment Violates "Acceptable" Behavior; Self-Destruct TBDs; Store Requirements in a Database
Comments
Do not do GUI without fast prototyping tools, else stay with TUI! (No tools, no GUI)
Whys and assumptions provide metaknowledge

Design
Transition from Requirements to Design Is Not Easy; Trace Design to Requirements; Evaluate Alternatives; Design Without Documentation Is Not Design; Encapsulate; Don't Reinvent the Wheel; Keep It Simple; Avoid Numerous Special Cases; Minimize Intellectual Distance; Keep Design Under Intellectual Control; Maintain Conceptual Integrity; Conceptual Errors Are More Significant Than Syntactic Errors; Use Coupling and Cohesion; Design for Change; Design for Maintenance; Design for Errors; Build Generality into Software; Build Flexibility into Software; Use Efficient Algorithms; Module Specifications Provide All the Information the User Needs and Nothing More; Design Is Multidimensional; Great Designs Come from Great Designers; Know Your Application; You Can Reuse Without a Big Investment; "Garbage In, Garbage oUt" Is Incorrect; Software Reliability Can Be Achieved Through Redundancy



Comments
"Garbage In, Garbage oUt" Is Incorrect (?): design garbage rejection (checks, PbC ...) The world is a mess, build a garden.
Do not outclever yourself; Be humble (but ask for a raise!)
Transition from Requirements to Design Is Not Easy : The whole industry is taking more than 20 years to learn this: OO, XP, Agile, Declarative, Iterative, Incremental ... But then it goes against the pay structure ... So real designers disappear
No evaluation of alternatives means a lemming is following a script
KISS: If we wanted complex, we would be doing it like always (without sw help)
Design for change: YAGNI, for you ...
Build Generality into Software: grow it!
Use Efficient Algorithms: Oh, but I would need education for that ...
The interface (in all senses) is the sw (to non-builders)

Coding
Avoid Tricks; Avoid Global Variables; Write to Read Top-Down; Avoid Side-Effects; Use Meaningful Names (for meaningful stuff); Write Programs for People First; Use Optimal Data Structures; Get It Right Before You Make It Faster; Comment Before You Finalize Your Code; Document Before You Start Coding; Hand-Execute Every Component; Inspect Code; You Can Use Unstructured Languages; Structured Code Is Not Necessarily Good Code; Don't Nest Too Deep; Use Appropriate Languages; Programming Language Is Not an Excuse; Language Knowledge Is Not So Important; Format Your Programs; Don't Code Too Soon


Comments
Write Programs for People First: This is the most important principle for coders (but choose your readers !)
Use Optimal Data Structures: requires education
Get It Right Before You Make It Faster: premature optimization
You Can Use Unstructured Languages: or untyped, or hacked (just be knowledgeable of the trade offs)
Don't Code Too Soon: true, but (for sweatshops) neither too late!
Removing lines of code is Good.


Testing
Trace Tests to Requirements; Plan Tests Long Before It Is Time to Test; Don't Test Your Own Software; Don't Write Your Own Test Plans; Testing Exposes Presence of Flaws; Though Copious Errors Guarantee Worthlessness, Zero Errors Says Nothing About the Value of Software; A Successful Test Finds an Error; Half the Errors Found in 15 Percent of Modules; Use Black-Box and White-Box Testing; A Test Case Includes Expected Results; Test Invalid Inputs; Always Stress Test; The Big Bang Theory Does Not Apply; Use McCabe Complexity Measure; Use Effective Test Completion Measures; Achieve Effective Test Coverage; Don't Integrate Before Unit Testing; Instrument Your Software; Analyze Causes for Errors; Don't Take Errors Personally

Comments
Nowadays, no unit testing means component coding is unfinished
Untested sw is worth less than zero: some people get away with making users the testers ...
You only learn from mistakes (if you ever learn).


Management
Good Management Is More Important Than Good Technology; Use Appropriate Solutions; Don't Believe Everything You Read; Understand the Customers' Priorities; People Are the Key to Success; A Few Good People Are Better Than Many Less Skilled; Listen to Your People; Trust Your People; Expect Excellence (and be a model); Communication Skills Are Essential; Carry the Water; People Are Motivated by Different Things; Keep the Office Quiet; People and Time Are Not Interchangeable; There Are Huge Differences Among Software Engineers; You Can Optimize Whatever You Want; Collect Data Unobtrusively; Cost Per Line of Code Is Not Useful; There Is No Perfect Way to Measure Productivity; Tailor Cost Estimation Methods ; Don't Set Unrealistic Deadlines; Avoid the Impossible; Know Before You Count; Collect Productivity Data; Don't Forget Team Productivity ; LOC/PM Independent of Language; Believe the Schedule; A Precision-Crafted Cost Estimate Is Not Foolproof; Reassess Schedules Regularly; Minor Underestimates Are Not Always Bad; Allocate Appropriate Resources; Plan a Project in Detail; Keep Your Plan Up-to-Date; Avoid Standing Waves; Know the Top 10 Risks; Understand Risks Up Front; Use an Appropriate Process Model; The Method Won't Save You; No Secrets for Miraculous Productivity Increases; Know What Progress Means; Manage by Variance; Don't Overstrain Your Hardware; Be Pessimistic About Software Evolution; The Thought That Disaster Is Impossible Often Leads to Disaster; Do a Project Postmortem

Comments
Don't Believe Everything You Read: This is the most important principle for managers
People trumps process, technology and rules
Use Appropriate Solutions: use proportional force
There Are Huge Differences Among Software Engineers: but it is in the interest of many (bad ones, the paying structure, friends) to hide this; but please, at least, remove the negative producers.
You Can Optimize Whatever You Want: but optimizing something may mean not optimizing something else.
Collect Data Unobtrusively: do not try to turn everybody into bean counters.
Cost Per Line of Code Is Not Useful: If you do not agree, then YOU are not useful.
Don't Set Unrealistic Deadlines: they are not deadlines, they are just ignored (maybe silently).
Know Before You Count: apples and oranges may not add up.
Nothing is foolproof: fools are the most intelligent agents of entropy in the Universe.
Be Pessimistic About Evolution: (surprise) entropy wins in the end.
The Method Won't Save You: Only people save (amen).
Manage by Variance: be a change detector and presenter.
(non natural) Disaster: requires multiple failures in a chain: if it happens, MANY things where done wrong. Some people really believe things sort out by themselves!


Product Assurance
Product Assurance Is Not a Luxury; Establish SCM Procedures Early; Adapt SCM to Software Process; Organize SCM to Be Independent of Project
Management; Rotate People Through Product Assurance; Give Every Intermediate Product a Name and Version; Control Baselines; Save Everything; Keep Track of Every Change; Don't Bypass Change Control; Rank and Schedule Change Requests; Use Validation and Verification (V&V) on Large
Developments


Evolution
Software Will Continue to Change; Software's Entropy Increases; If It Ain't Broke, Don't Fix It; Fix Problems, Not Symptoms; Change Requirements First; Prerelease Errors Yield Postrelease Errors; The Older a Program, the More Difficult It Is to Maintain; Language Affects Maintainability; Sometimes It Is Better to Start Over; Renovate the Worst First; Maintenance Causes More Errors Than Development; Regression Test After Every Change; Belief That a Change Is Easy Makes It Likely It Will Be Made Incorrectly; Structuring Unstructured Code Does Not Necessarily
Improve It; Use Profiler Before Optimizing; Conserve Familiarity; The System's Existence Promotes Evolution

Comments
Software rots (we have not much self-modifying sw and the world keeps changing).
Language (use) Affects Maintainability: repeat until death! Beware of fashion.
Sometimes It Is Better to Start Over: Sometimes It would have been Better to Start Over
Renovate the Worst First: Low hanging fruit tastes great!
Belief That a Change Is Easy Makes It Likely It Will Be Made Incorrectly: this can only be learned by mistake :)
The System's Existence Promotes Evolution: Real sw engineers are social engineeers (of a sort).