Fix#2897. We have been relying on iconv/mb_convert_encoding to detect invalid UTF-8, but all techniques designed to validate UTF-8 seem to accept FFFE and FFFF. This PR explicitly converts those characters to FFFD (Unicode substitution character) before validating the rest of the string. It also substitutes one or more FFFD when it detects invalid UTF-8 character sequences.
A comment in the code being change stated that it doesn't handle surrogates. It is right not to do so. The only case where we should see surrogates is reading UTF-16. Additional tests are added to an existing test reading a UTF-16 Csv to demonstrate that surrogates are handled correctly, and that FFFE/FFFF are handled reasonably.
* Additional Support for Chart DataSeriesValues
Fix#2863. DataSeriesValues now extends Properties, allowing it to share code in common with Axis and Gridlines. This causes some minor breakages; in particular line width is now initialized to null instead of Excel's default value, and is specified in points, as the user would expect from Excel, rather than the value stored in Xml.
This change:
- adds support for 1 or 2 marker colors.
- adds support for `smoothLine` to DataSeriesValues.
- will determine `catAx` or `valAx` for Axis based on what is read from the Xml when available, rather than guessing based on format. (Another minor break.)
- reads `formatCode` and `sourceLinked` for Axis.
- correct 2 uses of `$plotSeriesRef` to `$plotSeriesIndex` in Writer/Xlsx/Chart.
- pushes coverage over 90% for Chart (88.70% overall).
* Update Change Log
I had updated previously but forgot to stage the member.
* Adopt Some Suggestions
Incorporate some changes suggested by @bridgeplayr.
* Use ChartColor for DSV Fill And Font Text
DataSeriesValues Fill could be a scalar or an array, so I saved it till last.
* Some Final Cleanup
No code changes.
Illustrate even more of the new features in existing sample files.
Deprecate *_ARGB in Properties/ChartColors in favor of *_RGB, because it uses only 6 hex digits. The alpha value is stored separately.
This one class consumes a lot of space in Phpstan baseline. The problem is that it is an interface to Jpgraph, which is not maintained in Composer. This means that we have to disable tests involving this module, since we are dealing with very old code in our test suite. This means that we are very unlikely to do any work on this member, so the code error reports are more of a distraction than anything else. Remove them for now, restoring them if we ever solve this problem.
Unit testing now results in 100% coverage for Axis and Properties. All the properties in methods in Gridlines were more or less duplicated in Axis, and these duplications are moved to the common ancestor Properties. So, there isn't anything left in Gridlines. PhpSpreadsheet Chart is now over 85% covered (it was below 35% until recently).
Properties are in many cases set to default to null/null-string, rather than the default values they receive from Excel, and are not written to Xml if unchanged. This is consistent with how Excel behaves. A new property `crossBetween` is added to Axis, and, with support for that added to Xlsx Reader and Writer, some minor Sample peculiarities are corrected, in particular, the charts were sometimes slightly truncated on the left and right edges.
* Add Support to Chart/Axis and Gridlines for Shadow
Continuing the work from #2865. Support is added for Shadow properties for Axis and Gridlines, and Glow and SoftEdges are extended to Gridlines. Tests are added. Some chart tests are moved from Reader/Xlsx and Writer/Xlsx so that most chart tests are under a single directory.
This is a minor breaking change. Since the support for these properties was just added, it can't really affect much in userland. Some properties had been stored in the form which the XML requires them rather than as the user would enter them to Excel. So, for example, setting the Glow size to 10 points would have caused it to be stored internally as 127,000. This change will store the size internally as 10, obviously making the appropriate conversion when reading from or writing to XML. This makes unit tests much simpler, and I think this is also what a user would expect, especially considering the difficulties in keeping track of the trailing zeros.
* More Tests
Confirm value change between internal and xml.
* Still More Tests
Add a little more coverage, and use a neat trick suggested by @MarkBaker in the discussion of PR #2724 to greatly simplify MultiplierTest.
* Add Support to Chart/Axis for Glow/SoftEdges
Chart is very under-covered in unit tests. I was hoping to just add some tests and be done with it, but that just won't suffice - many code changes are required. Adding Glow properties for Axis turned out to be a real mixed bag - no support in Xlsx/Reader at all, support in Xlsx/Writer ... for the X-axis only. So we'll just inch forward in many stages.
This change does not address other Axis properties (Fill, Shadow, Line, LineStyle, and maybe others, and their sub-properties). On my to-do list.
GridLines, and maybe other Chart objects, also should support these properties. This change does not address those. On my to-do list.
No support is added for spreadsheet formats other than Xlsx.
* Refactoring
This should make the code easier to follow, and I hope it will also be extensible to other object types (e.g. GridLines).
* More Refactoring
Make scheme/srgb easier to handle.
* Fix Typo
In a very rarely used function.
Continuing the work from #2828, #2841, #2846, and #2852. This is probably my last change in this area for a while.
Bubble charts can have bubbles of different sizes. Phpspreadsheet had not supported this. Openpyxl comes with sample code to generate such a chart. I was especially drawn to that solution because its namespace usage would have been unexpected before 2852. And it turned out to come with other surprises - use of absolute paths in the .rels files (PhpSpreadsheet expected only relative), use of a one-cell anchor to place the chart (PhpSpreadsheet expected two-cell anchor or absolute positioning), plaintext in the legend (Phpspreadsheet expected RichText), no cached values for chart data. Excel handles the file okay, and this PR makes sure PhpSpreadsheet does as well. This file is now Samples/Templates/32readwriteBubbleChart2, and is used for both generating a sample output file and in formal tests. A new sample in the 33* series demonstrates how to code these.
* Namespacing Part 3 - Charts
Charts still need some work, but they are now in good enough shape that the code that reads them can handle namespacing properly. Rather than a set of static routines, Reader/Xlsx/Chart needs to be changed to a regular object, with the relevant namespaces passed in the constructor. I am still looking for a test spreadsheet with non-standard namespacing for a full-blown test, but, in the meantime, the existing tests and samples will suffice. When I find a test case, I will certainly add it.
The resulting code is actually simpler than it had been because of the elimination of the need for a NamespacesMeta variable in all the function signatures. Because of that, enough errors disappeared from Phpstan that I decided to just finish that job - its baseline file now dips below 5,000 lines.
* Update Change Log
Need to document many chart changes.
* Eliminate Corruption in Surface Chart Samples
Fix#2763 (bubble charts were fixed earlier).
* Improvement in Testing for Numeric Values
The original proposal was to test for numeric values by checking whether the format was NUMERIC or DATE. This is pretty inflexible, and left open a wide gap concerning what to do with GENERAL. Changed to allow the user to set a `numeric` property at the same time as `axisNumberProperties`, eliminating these concerns. User can set property when chart is created (new test member Charts32CatAxValAxTest), or can set it when spreadsheet with chart is read (new method testCatAxValAx in Charts32XmlTest). Sample 33_Chart_create_scatter2 creates a slightly wider chart when `numeric` is set to true than it does when unset, but chart is otherwise the same.
Corrected a bug which occured twice in Chart. Methods getChartAxisX and getChartAxisY created a new Axis object if the chart pointer was null, but neglected to attach the new object to the chart. Thus, subsequent calls to the methods return different objects. Code is changed to attach the new Axis to the Chart.
* More Chart Fixes
Taking up where #2828 left off. Most of the following changes are demonstrated in 32readwriteChartWithImages1:
- Adds support for "scheme" colors (because rgb, theme, and index colors just weren't enough for Excel) for DataSeriesValues. See issue #2299.
- For chart titles (including axis labels), rather than a font name, Excel supplies a 3-fold series of font names for Latin, East Asian, and Complex Scripts. New properties `latin`, `eastAsian`, and `complexScript` are added to the Font class. I frankly have no idea how, or even if, you can set these in Excel; my test case (sample 32readwriteScatterChart7) is a result of manually editing the XML.
- Add support for subscript/superscript to chart titles. This requires a new property `baseLine` in Font (positive=superscript negative=subscript baseline value says how high/low).
- Support for underscore with different scheme color than its text, using a new string property `uSchemeClr` in Font.
- Support for extra options for strikethrough, using a new string property `strikeType` in Font.
- Support for extra options for underscore type, using the existing string property `underline` in Font.
- I do not anticipate that any of the new Font properties will be used except for chart titles.
- If no default font overrides are found for a Rich Text element in chart titles, and no explicit font overrides are found for a Run under such an element, the font element of the Run is set to null.
- PhpSpreadsheet will always write a tag `a:pPr` and, underneath that, an empty tag `a:defRPr`, for default font settings for chart titles and axis labels. Combined with the previous bullet item, this will prevent PhpSpreadsheet from inadvertently overriding the Excel defaults (18 point bold Calibri for chart title, 10 point bold Calibri for axis labels).
- Axis labels will now be written to XML in the same manner as chart titles. Among other considerations, this means that they can now have colors. Fix#2700. Supersedes PR #2701. Demonstrated in sample 32readwriteStockChart5.
* Fix Some Chart Corruption
Fix#2817, where @bridgeplayr gives an excellent description of the problem and how it should be solved.
* Fix Bubble Charts
Sample produced corrupt output - see issue #2763. After a lot of research, solution was just re-ordering of parameters in a single function call.
Bubble 3D had not been supported at all. It is now.
Surface Charts remain corrupted.
* Some Fixes for Scatter Charts
Chart issues have been pouring in recently. This is a partial response to issue #2762. It implements "no joins" for scatter charts, as well as having the reader and writer handle "point size", "line width", and "color" for markers. A new boolean property `scatterLines`, with setter and getter, is added to DataSeriesValues to handle joins (default is true which means scatter plot points *are* joined by lines). Some, but not yet all, default font properties for the chart title are handled (color and, surprisingly, font name present challenges).
With these changes, sample 32readwriteScatterChart1.xlsx now looks closer to its source. There are still some differences (x-axis changes), but I think this change is already large enough. I can work on the other problems later.
The code for reading charts has not yet been converted to be namespace aware. Having a tiny island of aware code in a sea of unaware makes no sense to me, so some of the new code is likewise unaware. I hope to be able to get to it eventually, but, among other considerations, it is difficult to generate suitable test cases.
* Add Formal Tests
Essentially the same as the corresponding Samples, but with formal assertions.
* Clean Up Some Code in Reader/Xlsx/Chart
Having added code to support default font attributes as well as element-specific font attributes for chart captions, there was duplicated code between the default and specific sections. I hope that this PR makes the code easier to follow.
* Add Support for Font Name and Color to XLSX Chart Titles
XML layout for these in new files differs from what program was expecting. Not sure if program expectations were wrong, or if this is a change to Excel since initial development.
* Minor Improvement
Handle theoretical case where Chart title has text but no font information.
* Support Bezier Curve and Scaling of X-Axis on Scatter Plot
For Bezier, need to specify `<c:smooth>` tag in addition to already supplied `<c:scatterStyle val="smoothMarker">`
For X-Axis, scatter needs to supply both X and y axis as `<c:valAx>` rather than `<c:catAx>` for X.
* Real Errors Identified in Calculation by Scrutinizer
Before Scrutinizer broke, I took a look at the remaining 43 errors which it categorized as 'major'. Most of these were false positives, but, in the case of Calculation and Reader/Xlsx/Chart, I was able to determine that its analysis of some of the problems was correct. There is little point addressing the false positives until it starts working again, but we should fix the real errors.
This PR addresses the real errors in Calculation.
- A test for `$pCellParent` should have been a test for `$pCellWorksheet`.
- A test for `$operand1Data['reference']` should have been a test for `$operand1Data['value']`.
- A test for `$operand2Data['reference']` should have been a test for `$operand2Data['value']`.
* Fix Attempt to Erroneously Call trim on Array
Fix#2780. Warning message is being issued when getting calculated value of cell with value `=INDIRECT(ADDRESS(ROW(),COLUMN()-2))/$C$4`. This appears to be the case for all recent (and probably not so recent) releases; it is not a result of changes to the code base. Fix added to this PR because the erring section of code was proximate to code already changed in the PR. Test added.
* Minor Code Changes
Apply some suggestions from @MarkBaker
PHP 8.2 is supposed to deprecate the use of `['self', 'functionname']` for callables, suggesting the use of `[self::class, 'functionname']` instead. We made this change in a recent PR, and, while I'm thinking about it, I'll fix the remaining 2 modules with this construction. Vlookup is already adequately covered in unit tests. Reader/Xls/MD5 is not; a unit test is added.
Fix#2768. DateFormatter handles only one of six special formats for time intervals `[h] [hh] [m] [mm] [s] [ss]`. This PR extends support to the rest. There should be no more than one of these in any format string. Although it certainly could make sense to treat `[d] [dd]` in the same manner, Excel does not seem to support those.
Interesting observations - hours and minutes are truncated (presumably because they may be followed by minutes and seconds), but seconds are rounded. Also, there are some floating point issues, which fortunately showed up for the example in the original issue. There, the time interval was 1.15, which should evaluate to a minutes value of 1656 (as it does in Excel). However, on my system it evaluated to 1655 because of a rounding error in the 13th decimal place. To overcome this, values are rounded to 10 decimal places before truncating.
I suspect that scrutiniser may complain that the pass-by-reference values in an expression like `sscanf($coord, '%[A-Z]%d', $column, $row)` don't exist; but PHP handles that without issue, creating the variables as needed, and phpstan has no problems with that, so scrutiniser shouldn't treat it as an issue. There's no point in adding code (even if it's just pre-defining call-by-reference arguments) when it's unnecessary overhead.
Dependabot submitted PRs #2719 and #2720 to upgrade Phpstan. As with most Phpstan upgrades, there are new error messages; this PR is an attempt to fix all 58 of the new problems in order to allow the upgrade to proceed.
Most of these fixes involve the addition of doc-block type annotations, often involving the assignment of the 'objectionable' portion of the statement to a new variable. Some use explicit casting when I am sure that's safe. Some (Reader/Ods) involve defeating result caching by Phpstan.
* Add editAs Property for 2-cell Anchor Drawings
This change builds on PR #2532 (@naotake51 as PR #2532), using ideas from PR #2237 (@AdamGaskins), which has had changes requested for several months. It covers a lot of the same ground as 2532. In Excel, two-cell anchor drawings can be edited as "twocell", "onecell", or "absolute". This PR adds support for those options, with a sample file that demonstrates the difference in addition to unit tests. Several other tests are added to improve the spotty coverage for Drawings.
There have been several other tickets referencing two cell anchors, including issue #1159 and PR #1160 (@sgarwood, who also added support for editAs), PR #643, and issue #126, all now closed but not necessarily entirely resolved. I will try to ensure that those tickets are addressed with this one.
And, in trying to make sure 1160 is covered, I stumbled upon a bug. If you use the same image resource to create two+ memory drawings, the MemoryDrawing destructor for the first will cause the rest to generate a very long warning message. This is not a problem for Php8+, only for Php7-. I have suppressed the message in the MemoryDrawing constructor. 1160 went stale due to an unresolved test error, but I don't think this was the problem. At any rate, its test works now.
* Scrutinizer
It reported 1 minor issue (fixed normally), and 2 major. One is fixed with a kludge. The other is a case where Scrutinizer's analysis is just wrong, and I can't figure out a kludge. But I was able to add an annotation (the first time I've managed to get one past phpcs/php-cs-fixer). We'll see.
These changes have already been implemented twice, and been regressed twice. I'll try once more (with a different approach), then give up ...
As configured, Phpstan running under Php7 reports no errors. However, running under Php8, it reports 100 (!) errors. The vast majority of these are due to two reasons:
- renaming parameters in Php builtin functions in preparation for named parameters.
- using the new class GdImage rather than type resource as the argument type for many image-based functions.
Regardless of the cause, this will be a problem sooner or later. This PR is an attempt to get ahead of that problem. For source members, it mostly adds annotations or updates doc-blocks. Only 2 members have changes to executable code, and these are very minor - BitWise and Writer/Xlsx. For test members, all baseline errors are deleted and the code is fixed. Php7 and Php8 both report no errors with this configuration.
Initial implementation of Excel's tables feature (i.e. Select Home >
Format as Table in Excel App).
Tables are similar to AutoFilter but tables have other advantages like
named ranges, easy formatting, totals row and header row with filter.
Tables can also be converted to charts and pivot tables easily.
Usage:
$table = new Table();
$table->setName('Sales_Data');
$table->setRange('A1:D17');
$spreadsheet->getActiveSheet()->addTable($table);
In this Commit:
- Added Table API with initial support for header and totals row.
- Added complete styling options for Table.
- Added Xlsx Writer for Table.
- Added samples.
- Covered with unit tests.
To be done:
- Filter expressions similar to AutoFilter.
- Precalucate formulas for totals row (Check sample 2).
- Table named ranges in formulas and calculation.
* Add two cell anhor drawing.
* Add "Support for two cell anchor drawing of images." to CHANGELOG.md
* Add pull-request link to "Support for two cell anchor drawing of images." of CHANGELOG.md
You can set up a conditional style to, say, apply to cells equal to boolean values. For such conditions, the Excel XML specifies `TRUE` or `FALSE`. It is noteworthy that false matches empty cells as well as FALSE, but not 0; similarly TRUE does not match 1. The Xlsx Writer just casts these values to string, which will not work properly. The Xlsx Reader treats the values as strings, so it won't work properly either. This PR corrects both. Also the doc blocks in Style/Conditional allow bool in some places, but not in others; these are corrected but no executable code is changed there.
oleibman
MarkBaker
Mark Baker
aswinkumar863
aswinkumar863
naotake51