Was broken by 82b521ae, as we have
positioning data by then and can't use the output buffer.  Ouch!

TODO: Documentation.

Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=71445

Fixes test-shape, and:
https://code.google.com/p/chromium/issues/detail?id=497578

Separate the loops for the two cases of replacing with space
and deleting.  For deleting, use the out-buffer machinery.

Needed for upcoming cluster merge fix.

After 763e5466, one doesn't
need to set flags for different pieces of text.  The flags now
are something the client sets up once, depending on how it
actually uses the buffer.  As such, don't clear it in
clear_contents().

Tests updated.

We can't really resize buffer and continue in this shaper as we are
using the scratch buffer for string_ref and log_cluster.  Restructure
shaper to retry from (almost) scratch.

With this change, we now by default replace broken UTF-8/16/32 bits
with U+FFFD.  This can be changed by calling new API on the buffer.
Previously the replacement value used to be (hb_codepoint_t)-1.

Note that hb_buffer_clear_contents() does NOT reset the replacement
character.

See discussion here:

https://github.com/behdad/harfbuzz/commit/6f13b6d62daae4989e3cc2fe4b168e5c59650964

New API:

  hb_buffer_set_replacement_codepoint()
  hb_buffer_get_replacement_codepoint()

Previously we were only setting this in hb_buffer_clear_contents(),
but set_length(0) is a valid way to reinitialize buffer to use with
new text.

Fixes last of scratch alignment warnings in hb-coretext.

Previously we only supported recursive sublookups with
ascending indices.  We were also not correctly handling
non-1-to-1 recursed lookups.

Fix all that!

Fixes the three tests in test/shaping/tests/context-matching.tests,
which were derived from NotoSansBengali and NotoSansDevanagari
among others.

Using a ragel machine.

API additions:

	hb_segment_properties_t
	HB_SEGMENT_PROPERTIES_DEFAULT
	hb_segment_properties_equal()
	hb_segment_properties_hash()

	hb_buffer_set_segment_properties()
	hb_buffer_get_segment_properties()

	hb_ot_layout_glyph_class_t

	hb_shape_plan_t
	hb_shape_plan_create()
	hb_shape_plan_create_cached()
	hb_shape_plan_get_empty()
	hb_shape_plan_reference()
	hb_shape_plan_destroy()
	hb_shape_plan_set_user_data()
	hb_shape_plan_get_user_data()
	hb_shape_plan_execute()

	hb_ot_shape_plan_collect_lookups()

API changes:

	Rename hb_ot_layout_feature_get_lookup_indexes() to
	hb_ot_layout_feature_get_lookups().

New header file:

	hb-shape-plan.h

And a bunch of prototyped but not implemented stuff.  Coming soon.
(Tests fail because of the prototypes right now.)

I don't expect anybody using hb_buffer_add(), so this shouldn't break
anyone's code.

New API:

	hb_buffer_flags_t

	HB_BUFFER_FLAGS_DEFAULT
	HB_BUFFER_FLAG_BOT
	HB_BUFFER_FLAG_EOT
	HB_BUFFER_FLAG_PRESERVE_DEFAULT_IGNORABLES

	hb_buffer_set_flags()
	hb_buffer_get_flags()

We use the BOT flag to decide whether to insert dottedcircle if the
first char in the buffer is a combining mark.

The PRESERVE_DEFAULT_IGNORABLES flag prevents removal of characters like
ZWNJ/ZWJ/...

Which is like _reset(), but does NOT clear unicode-funcs.

We should make it clear that we don't want output buffer in this case,
otherwise buffer->backtrack_len() would be wrong.

To be used for a variety of purposes.  We save up to five characters
in each direction.  No public API changes, everything is taken care
of already.  All clients need to do is to call hb_buffer_add_utf* with
the full text + segment info (or at least some context) instead of
just passing in the segment.

Various operations (hb_buffer_reset, hb_buffer_set_length,
hb_buffer_add*) automatically reset the relevant contexts.

And hb_buffer_content_type_t and enum values.

No panic, we reeally insert dotted circle when it's absolutely broken.

Fixes most of the dotted-circle cases against Uniscribe. (for Devanagari
fixes 80% of them, for Khmer 70%; the rest look like Uniscribe being
really bogus...)

I had to make a decision.  Apparently Uniscribe adds one dotted circle
to each broken character.  I tried that, but that goes wrong easily with
split matras.  So I made it add only one dotted circle to an entire
broken syllable tail.  As in: "if there was a dotted circle here, this
would have formed a correct cluster."  That works better for split
stuff, and I like it more.

To be used in places we access buffer vars...

https://bugs.freedesktop.org/show_bug.cgi?id=53233

Should give us some performance boost.

Just in case, for the times we do out-of-bounds access.

jk

Detected by clang and lots of cmdline options.